-
Page updated
-
Sources checked
-
File sizes from Hugging Face and Ollama, read October 17, 2026
-
Hardware recipes as NVIDIA, Unsloth, Ollama and LM Studio publish them; no speeds invented
-
Jump to the checker ↓ Nemotron 3.5 is NVIDIA's open-weight model family released on October 15, 2026. Two models run locally: Bolt 74B-A9B, a mixture-of-experts vision-language model built to run on one machine in about 24 GB with NVIDIA's own 2-bit GGUF, and its parent Super VL 120B-A12B, a 120B model with a 1M-token context aimed at DGX Spark and data-center GPUs. Both are hybrid Mamba-2 / transformer models with text, image and video input, under the OpenMDW-1.1 license.
Short answer: Bolt's official GGUF is 22.0 GB (2-bit), but NVIDIA's own llama.cpp recipes start at a 32 GB GPU (RTX 5090) or a 64 GB RTX Spark, with 64K context; NVIDIA's 256K-context recipe is for a 48 GB GPU or a 128 GB Spark. Unsloth budgets about 32 GB of RAM or unified memory for short contexts. NVIDIA publishes no recipe for 24 GB cards such as the RTX 4090. Super VL does not fit one consumer GPU: Unsloth's GGUFs need about 64 GB of total memory (RAM + VRAM, or unified memory) for 1-bit, 96 GB for NVFP4 and 128 GB for UD-Q4_K_XL (4-bit).
Match your hardware to a published Nemotron 3.5 route #
[Open in a full page ↗](https://tools.localrigcheck.com/nemotron-3-5/)
private checker…
- Developer
- NVIDIA [Official](#src-1)
- Params · Nemotron 3.5 Bolt 74B-A9B
- 74B total, 9B active per token [Official](#src-1)
- Params · Nemotron 3.5 Super VL 120B-A12B
- 120B total, 12B active [Official](#src-2)
- Context · Nemotron 3.5 Bolt 74B-A9B
- up to 256K tokens (262,144) [Official](#src-1)
- Context · Nemotron 3.5 Super VL 120B-A12B
- up to 1M tokens (1,048,576) Official
- Release
- October 15, 2026 on Hugging Face (nvidia organization, not gated) Official
- License
- OpenMDW-1.1 [Official](#src-1)
- llama.cpp · Nemotron 3.5 Bolt 74B-A9B
- NVIDIA's own llama.cpp recipes start at a 32 GB GPU (RTX 5090) [Official](#src-1)
How much VRAM does Nemotron 3.5 Bolt need? #
File sizes as Hugging Face and Ollama list them (decimal GB, read October 17, 2026). NVIDIA designed Bolt for its own 2-bit GGUF: the experts were trained for 2-bit (quantization-aware distillation), and NVIDIA describes the result as fitting in 24 GB. Bits per weight = size × 8 ÷ 74B parameters. Context memory comes on top.
| Format | File size | Bits/weight | Runs with | Source |
|---|---|---|---|---|
| BF16 (official weights) | 149.8 GB | 16.2 | vLLM (NVIDIA: for customization and post-training) | NVIDIA |
| 24GB packed (official, safetensors) | 24.1 GB | 2.6 | vLLM | NVIDIA |
| 24GB GGUF (official, IQ2_XXS experts) | 22.0 GB | 2.4 | llama.cpp | NVIDIA |
| + vision file (mmproj, BF16) | 1.7 GB | not a 74B-weight file | llama.cpp, for image and video input | NVIDIA |
| + LiLiCorr draft model (GGUF) | 1.0 GB | not a 74B-weight file | llama.cpp speculative decoding (optional) | NVIDIA |
| Ollama nemotron-3.5-bolt (latest, 74b-a9b-iq2) | 25 GB | not stated by Ollama | Ollama, text and image | Ollama library |
| Unsloth BF16 GGUF | 148.2 GB | 16.0 | llama.cpp | Unsloth |
| Unsloth Q8_0 | 78.9 GB | 8.5 | llama.cpp | Unsloth |
| Unsloth UD-Q6_K_XL | 72.4 GB | 7.8 | llama.cpp | Unsloth |
| Unsloth UD-Q5_K_XL | 58.9 GB | 6.4 | llama.cpp | Unsloth |
| Unsloth UD-Q4_K_XL | 50.2 GB | 5.4 | llama.cpp | Unsloth |
| Unsloth UD-Q3_K_XL | 39.0 GB | 4.2 | llama.cpp | Unsloth |
| Unsloth UD-IQ3_XXS | 31.6 GB | 3.4 | llama.cpp | Unsloth |
| Unsloth UD-Q2_K_XL | 28.8 GB | 3.1 | llama.cpp | Unsloth |
| Unsloth UD-IQ2_M | 24.6 GB | 2.7 | llama.cpp | Unsloth |
| Unsloth IQ2_XXS / UD-IQ2_XXS | 22.0 GB | 2.4 | llama.cpp | Unsloth |
| Unsloth UD-IQ1_M | 20.3 GB | 2.2 | llama.cpp | Unsloth |
| Unsloth UD-IQ1_S | 18.5 GB | 2.0 | llama.cpp | Unsloth |
Bits per weight are file size over 74B parameters (RigCheck arithmetic); the files also hold layers that are not 2-bit (NVIDIA's technical blog: Mamba linear layers in NVFP4; embeddings, output head, attention and latent-MoE weights in Q8_0), so the average is above the experts' 2.06 bits. Unsloth's guide covers only the 22 GB IQ2_XXS file and says Bolt "is available in 1-bit and 2-bit"; the larger Unsloth files are listed in its repository, but its guide gives no memory figure or recommendation for them.
What hardware can run Nemotron 3.5 Bolt? #
| Hardware | What the sources say | Source |
|---|---|---|
| RTX Spark 128 GB, or a discrete RTX GPU with 48 GB+ (e.g. RTX PRO 6000, RTX 6000 Ada) | NVIDIA's "32GB recipe": llama.cpp with the 2-bit GGUF, vision file and draft model fully on the GPU, 262,144-token (256K) context, one request at a time | NVIDIA |
| RTX Spark 64 GB, or a discrete RTX GPU with 32 GB+ (e.g. RTX 5090) | NVIDIA's "28GB recipe": the same files, 65,536-token (64K) context, q8_0 KV cache, one request at a time | NVIDIA |
| 24 GB GPU (RTX 4090, RTX 3090) | No NVIDIA recipe. On paper the 22.0 GB GGUF plus the 1.7 GB vision file is 22.1 GiB of a 24 GiB card, about 1.9 GiB left for context, compute buffers and the display; adding the 1.0 GB draft model leaves about 1 GiB (RigCheck arithmetic, not a tested setup). Leaving out the vision file and draft model, using a short context, or keeping some layers in system RAM frees more; we found no published measurement. The RTX 3090 (Ampere) is also outside the GGUF model card's supported architectures (Ada Lovelace, Blackwell) | RigCheck |
| PC or Mac with 32 GB of RAM or unified memory | Unsloth: budget about 32 GB of RAM or unified memory for short-context use, 48–64 GB for more headroom; its table lists 24–32 GB for the 22 GB IQ2_XXS file. Longer contexts, images and video need more. The draft model adds about 1–2 GB. Unsloth's guide gives 128K as Bolt's maximum context (NVIDIA: 256K) | Unsloth guide |
| DGX Spark, or NVIDIA GPUs with vLLM | NVIDIA's technical blog: the 24GB packed build is optimized for DGX Spark with vLLM (and the GGUF for RTX Spark with llama.cpp). Recipe: 256K context, fp8 KV cache, one request at a time, a nightly vLLM image. The model card is inconsistent: its summary lists A100, H100 80 GB and Blackwell, but its compatibility line lists only Blackwell SM120/SM121 (RTX 50-series / RTX PRO and DGX Spark's GB10), not data-center Blackwell | NVIDIA |
NVIDIA writes that Bolt's architecture search "targets a deployment footprint below 24 GB at batch size 1 and 128K context", and its blog says the model fits "in 24GB of memory and run[s] alongside other applications" on RTX and DGX Spark, which have 64 or 128 GB of unified memory. NVIDIA's blog also says Bolt runs on GeForce RTX PCs but names no memory size; its published llama.cpp recipes for discrete GPUs start at 32 GB. The GGUF card's Quick Start uses 98,304 tokens (96K) with a q8_0 KV cache and names no hardware. Capacity is not compatibility: the official GGUF model card lists NVIDIA Ada Lovelace and Blackwell GPUs on Windows and Linux. For Mac, Unsloth's llama.cpp guide covers Metal builds; we found no source covering AMD GPUs (ROCm or Vulkan) for this model, and NVIDIA publishes no recipe for either.
How much VRAM does Nemotron 3.5 Super VL need? #
File sizes as Hugging Face and Ollama list them (decimal GB, read October 17, 2026; ggml-org's Super VL files October 20). Bits per weight = size × 8 ÷ 120B parameters. Context memory comes on top.
| Format | File size | Bits/weight | Runs with | Source |
|---|---|---|---|---|
| BF16 (official weights) | 248.9 GB | 16.6 | vLLM, SGLang | NVIDIA |
| NVFP4 (official weights) | 75.0 GB | 5.0 | vLLM, SGLang, TensorRT-LLM | NVIDIA |
| ggml-org BF16 / Q8_0 / NVFP4 / Q4_0 GGUF | 241.5 / 128.5 / 76.9 / 71.9 GB | 16.1 / 8.6 / 5.1 / 4.8 | llama.cpp | ggml-org |
| Unsloth Q8_0 | 131.6 GB | 8.8 | llama.cpp | Unsloth |
| Unsloth UD-Q6_K_XL | 121.0 GB | 8.1 | llama.cpp | Unsloth |
| Unsloth UD-Q5_K_XL | 110.7 GB | 7.4 | llama.cpp | Unsloth |
| Unsloth UD-Q4_K_XL | 86.9 GB | 5.8 | llama.cpp | Unsloth |
| Unsloth NVFP4 GGUF | 77.7 GB | 5.2 | llama.cpp | Unsloth |
| Unsloth UD-Q3_K_XL | 65.8 GB | 4.4 | llama.cpp | Unsloth |
| Unsloth UD-IQ3_XXS | 59.8 GB | 4.0 | llama.cpp | Unsloth |
| Unsloth UD-Q2_K_XL | 57.8 GB | 3.9 | llama.cpp | Unsloth |
| Unsloth UD-IQ2_M | 55.9 GB | 3.7 | llama.cpp | Unsloth |
| Unsloth UD-IQ2_XXS | 51.1 GB | 3.4 | llama.cpp | Unsloth |
| Unsloth UD-IQ1_M | 46.1 GB | 3.1 | llama.cpp | Unsloth |
Ollama nemotron-3.5-super : latest (120b) / q4_K_M / q8_0 / bf16 |
80–90 / 90 / 133 / 249 GB | not stated by Ollama | Ollama (latest is listed as a range and is a different build from q4_K_M) | Ollama library |
| Ollama MLX tags: nvfp4 / mxfp8 / mlx-bf16 | 80 / 130 / 249 GB | not stated by Ollama | Ollama on Apple silicon (MLX) | Ollama library |
Bits per weight are file size over 120B parameters (RigCheck arithmetic). NVIDIA's safetensors include the vision encoder and the MTP head; ggml-org's GGUFs keep them in separate files: vision (mmproj) 1.7 / 1.0 GB at BF16 / Q8_0, MTP head 8.0 / 4.3 / 2.3 GB at BF16 / Q8_0 / Q4_0, and a LiLiCorr draft model of 1.35 GB (Unsloth: mmproj 1.7 GB, LiLiCorr 1.29 GB). Each adds memory when loaded.
What hardware can run Nemotron 3.5 Super VL? #
Unsloth's table of total memory (RAM + VRAM, or unified memory) for its GGUF files with llama.cpp. These are Unsloth's planning estimates "for a modest context", including the OS, runtime, KV cache and vision file, not a RigCheck estimate.
| Quantization | GGUF size | Total memory | Source |
|---|---|---|---|
| UD-IQ1_M (1-bit) | 46.1 GB | 64 GB | Unsloth guide |
| UD-IQ2_XXS (2-bit) | 51.1 GB | 64 GB | Unsloth guide |
| UD-IQ2_M, UD-Q2_K_XL (2-bit), UD-IQ3_XXS (3-bit) | 55.9–59.8 GB | 80 GB | Unsloth guide |
| UD-Q3_K_XL (3-bit) | 65.8 GB | 96 GB | Unsloth guide |
| NVFP4 (4-bit) | 79 GB | 96 GB | Unsloth guide |
| UD-Q4_K_XL (4-bit) | 86.9 GB | 128 GB | Unsloth guide |
| UD-Q5_K_XL, UD-Q6_K_XL | 111–121 GB | 160 GB | Unsloth guide |
| Q8_0 (8-bit) | 132 GB | 192 GB | Unsloth guide | Unsloth's guide rounds the NVFP4 GGUF to 79 GB; its repository lists 77.7 GB. Unsloth recommends UD-Q4_K_XL as the best balance of accuracy (KL divergence) and disk space, with NVFP4 and UD-Q3_K_XL also reasonable.
| Hardware | What the sources say | Source |
|---|---|---|
| One gaming GPU (RTX 4090 24 GB, RTX 5090 32 GB) | Not on the card alone: the smallest GGUF is 46.1 GB. llama.cpp can split the model across system RAM and VRAM, so you need the total-memory row above in RAM + VRAM (Unsloth) | Unsloth guide |
| DGX Spark (128 GB) | NVIDIA: the NVFP4 weights with vLLM (text and image, 1M-token context, LiLiCorr or MTP drafting), SGLang, or TensorRT-LLM (128K context). Unsloth: a 128 GB unified-memory system fits its 4-bit memory budget | NVIDIA, Unsloth guide |
| RTX PRO 6000 (96 GB), RTX Spark | NVIDIA's NVFP4 card points local users on DGX Spark, RTX Spark and RTX PRO 6000 to llama.cpp (ggml-org GGUF) and Unsloth; it publishes no recipe of its own for these | NVIDIA |
| Mac with 64–192 GB of unified memory | Unsloth's table counts unified memory once (64 GB for 1-bit up to 192 GB for 8-bit) and its llama.cpp guide covers Metal. Ollama lists MLX builds for Apple silicon (nvfp4 80 GB, mxfp8 130 GB). LM Studio lists a minimum of 70 GB of system memory. NVIDIA names no Mac setup | Unsloth guide, Ollama library, LM Studio |
| Data-center GPUs | NVFP4: one H200, two H100s or one GB200 (vLLM, 1M-token context); BF16: four H100s or two GB200s (vLLM, SGLang) | NVIDIA |
Which runtimes run Nemotron 3.5 locally? #
| Runtime | Status (checked October 17, 2026) | Source |
|---|---|---|
| llama.cpp | Bolt: NVIDIA's own GGUF and recipes; build the latest llama.cpp from source. Super VL: GGUF files from ggml-org (the llama.cpp project's own organization) and Unsloth. Both take images through the mmproj file | NVIDIA, ggml-org, Unsloth guide |
| Ollama | nemotron-3.5-bolt (25 GB, 256K context) andnemotron-3.5-super (latest/120b at 80–90 GB, plus q4_K_M, q8_0 and bf16 tags, all with 1M context; MLX builds with no context listed), both text and image input |
Ollama library |
| LM Studio | Super VL has a model page ( nvidia/nemotron-3.5-super-vl , minimum 70 GB of system memory). NVIDIA's blog says Bolt is supported; we found no Bolt model page on October 17 |
LM Studio, NVIDIA |
| Unsloth Desktop | Runs both on macOS, Windows and Linux with Unsloth's GGUF files; for Bolt it adds the NVFP4 draft model by default | Unsloth guide |
| vLLM, SGLang, TensorRT-LLM | Super VL on DGX Spark and data-center GPUs (NVFP4 and BF16). Bolt's 24GB packed build runs on vLLM (a nightly image at release), which NVIDIA optimized for DGX Spark | NVIDIA |
How fast is Nemotron 3.5 locally? #
NVIDIA's one published local figure: Bolt reaches 84 tokens per second (83.99) on AIPerf SPEED-Bench-Coding with llama.cpp and its LiLiCorr draft model, on a 128 GB RTX Spark (Surface Laptop Ultra, Windows 11 on Arm); on the same machine an AA-AgentPerf Local replay took 600.80 seconds. NVIDIA's design target was at least 80 tokens per second NVIDIA technical blog. That is one machine with speculative decoding, not a figure for the 64 GB RTX Spark, an RTX 5090, a Mac or partial offload. We found no published local speed for Super VL; NVIDIA's "up to 5x faster" claim is a throughput comparison on the Artificial Analysis leaderboard, not a local token rate.
Measured Nemotron 3.5 on your own hardware, or tried and it failed? Send us the numbers: paste the benchmark output or log into the form (or email it). After checking it we add it to this page, labeled as a community report and shared under CC BY 4.0.
Nemotron 3.5 specs #
| Parameters | Bolt: 74B total, 9B active per token. Super VL: 120B total, 12B active Model cards |
|---|---|
| Architecture | Hybrid Mamba-2 / transformer mixture-of-experts with a C-RADIO v4-H vision encoder: 88 layers, of which 40 Mamba-2, 40 MoE and 8 attention (2 KV heads of size 128). Super VL: 512 routed experts, 22 active; Bolt was compressed from Super VL with 9 active config.json, model cards |
| Context length | Bolt: up to 256K tokens (262,144). Super VL: up to 1M tokens (1,048,576) Model cards |
| Context memory | Only the 8 attention layers keep a KV cache: about 8 KiB per token at 16-bit, so about 1.1 GB at 128K and 2.1 GB at 256K; Super VL at 1M about 8.6 GB. The Mamba-2 layers add a fixed state that does not grow with context; using the MTP head adds one more attention layer (RigCheck estimate from config.json) |
| Input / output | Text, images and video (MP4 up to 2 minutes) in; text out Model cards |
| Release | October 15, 2026 on Hugging Face (nvidia organization, not gated). NVIDIA says a smaller Bolt E4B (4 GB) follows on October 23 NVIDIA |
| License | OpenMDW-1.1; NVIDIA says the models are ready for commercial and non-commercial use Model cards |
Common setups #
The checker's own answer for 8 common setups (graphics cards with 32 or 64 GB of system RAM, then Macs), in the finder's quick mode (graphics cards: Bolt and Super VL on one card with the RAM given: NVIDIA's llama.cpp recipes at their own context (64K / 256K), else Unsloth's short-context memory budgets; Macs: Bolt and Super VL in unified memory: Unsloth's short-context memory budgets (NVIDIA names no Mac setup)). Answers are published route matches, not measured setups; "Unknown" means the checker has no route or source for that hardware. Your own setup: the checker.
| Nemotron 3.5: answers by setup | ||
|---|---|---|
| Setup | Nemotron 3.5 Bolt 74B-A9B | Nemotron 3.5 Super VL 120B-A12B |
| --- | --- | --- |
| GeForce RTX 3060 (12 GB) + 32 GB RAM | ! Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (published route match)This card is outside the architectures NVIDIA's GGUF card lists (Ada Lovelace, Blackwell), and NVIDIA publishes no recipe for it. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 32 GB. That is Unsloth's planning estimate, not a measured run on this hardware. | ✕ Below Unsloth's smallest memory budget for Super VL (64 GB) Unsloth plans at least 64 GB of RAM or unified memory for its smallest GGUF (UD-IQ1_M, 46.1 GB); llama.cpp can split the model across RAM and VRAM. This setup counts 12 GB of VRAM + 32 GB of RAM = 44 GB. |
| GeForce RTX 4060 Ti (16 GB) + 32 GB RAM | ! Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (published route match)NVIDIA publishes no recipe for a 16 GB card; its discrete-GPU recipes start at 32 GB. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 32 GB. That is Unsloth's planning estimate, not a measured run on this hardware. | ✕ Below Unsloth's smallest memory budget for Super VL (64 GB) Unsloth plans at least 64 GB of RAM or unified memory for its smallest GGUF (UD-IQ1_M, 46.1 GB); llama.cpp can split the model across RAM and VRAM. This setup counts 16 GB of VRAM + 32 GB of RAM = 48 GB. |
| GeForce RTX 3090 / 3090 Ti (24 GB) + 64 GB RAM | ! Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (published route match)This card is outside the architectures NVIDIA's GGUF card lists (Ada Lovelace, Blackwell), and NVIDIA publishes no recipe for it. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 64 GB. That is Unsloth's planning estimate, not a measured run on this hardware. | ! Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB) (published route match)Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 24 GB of VRAM + 64 GB of RAM = 88 GB. Longer contexts and video need more. |
| GeForce RTX 5090 (32 GB) + 64 GB RAM | ! Matches NVIDIA's 28GB llama.cpp recipe: 64K context, q8_0 KV cache, one request at a time (published route match)NVIDIA's 256K-context recipe needs an RTX Spark 128 GB or a 48 GB+ discrete GPU; this hardware gets the 65,536-token recipe. Build the latest llama.cpp from source first. No speed is published for this machine; NVIDIA's 84 tokens per second is from a 128 GB RTX Spark. | ! Matches Unsloth's 96 GB budget: UD-Q3_K_XL (65.8 GB) or NVFP4 (79 GB) (published route match)Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 32 GB of VRAM + 64 GB of RAM = 96 GB. Longer contexts and video need more. |
| Apple Silicon Mac (64 GB unified) | ! Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (after raising the Mac's GPU wired-memory limit) (published route match)NVIDIA names no Mac setup; Unsloth's llama.cpp guide covers Metal builds. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This unified memory is 64 GB. That is Unsloth's planning estimate, not a measured run on this hardware. On a Mac, raise the GPU wired-memory limit first. | ! Matches Unsloth's 64 GB budget: UD-IQ1_M (46.1 GB) or UD-IQ2_XXS (51.1 GB) (after raising the Mac's GPU wired-memory limit) (published route match)Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 64 GB of unified memory (counted once). Longer contexts and video need more. On a Mac, raise the GPU wired-memory limit first. |
| Mac with M5 Max, 40-core GPU (128 GB unified) | ! Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (after raising the Mac's GPU wired-memory limit) (published route match)NVIDIA names no Mac setup; Unsloth's llama.cpp guide covers Metal builds. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This unified memory is 128 GB. That is Unsloth's planning estimate, not a measured run on this hardware. On a Mac, raise the GPU wired-memory limit first. | ! Matches Unsloth's 128 GB budget: UD-Q4_K_XL (4-bit, 86.9 GB), Unsloth's recommended balance (after raising the Mac's GPU wired-memory limit) (published route match)Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 128 GB of unified memory (counted once). Longer contexts and video need more. On a Mac, raise the GPU wired-memory limit first. |
| Mac Studio M3 Ultra (256 GB unified) | ! Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (after raising the Mac's GPU wired-memory limit) (published route match)NVIDIA names no Mac setup; Unsloth's llama.cpp guide covers Metal builds. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This unified memory is 256 GB. That is Unsloth's planning estimate, not a measured run on this hardware. On a Mac, raise the GPU wired-memory limit first. | ! Matches Unsloth's 192 GB budget: Q8_0 (8-bit, 132 GB) (after raising the Mac's GPU wired-memory limit) (published route match)Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 256 GB of unified memory (counted once). Longer contexts and video need more. On a Mac, raise the GPU wired-memory limit first. |
| Mac Studio M3 Ultra (512 GB unified) | – Unknown No published Nemotron 3.5 route checked here for this device. | – Unknown No published Nemotron 3.5 route checked here for this device. |
Machine-readable: the same answers as JSON (CC BY 4.0, attribution "RigCheck").
Related hardware #
Graphics card and Mac pages where this page's checker gives an answer, smallest memory first: graphics cards with 64 GB of system RAM, Macs with their unified memory, in the finder's quick mode (published route match; each page lists every model).
- RTX 3060 12 GB : Nemotron 3.5 Bolt 74B-A9B: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory; Nemotron 3.5 Super VL 120B-A12B: Matches Unsloth's 64 GB budget: UD-IQ1_M (46.1 GB) or UD-IQ2_XXS (51.1 GB)
- RTX 4070 Ti : Nemotron 3.5 Bolt 74B-A9B: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory; Nemotron 3.5 Super VL 120B-A12B: Matches Unsloth's 64 GB budget: UD-IQ1_M (46.1 GB) or UD-IQ2_XXS (51.1 GB)
- RTX 4060 Ti 16 GB : Nemotron 3.5 Bolt 74B-A9B: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory; Nemotron 3.5 Super VL 120B-A12B: Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB)
- RTX 3090 : Nemotron 3.5 Bolt 74B-A9B: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory; Nemotron 3.5 Super VL 120B-A12B: Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB)
- RTX 4090 : Nemotron 3.5 Bolt 74B-A9B: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory; Nemotron 3.5 Super VL 120B-A12B: Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB)
- Mac with 32 GB : Nemotron 3.5 Bolt 74B-A9B: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (after raising the Mac's GPU wired-memory limit)
All graphics cards · All Macs · Any other machine
FAQ #
How much VRAM does Nemotron 3.5 Bolt need? #
The official 2-bit GGUF is 22.0 GB, plus 1.7 GB for image input and 1.0 GB for the optional draft model. NVIDIA's llama.cpp recipes run it fully on a 32 GB GPU (64K context) or a 48 GB GPU (256K context), or on an RTX Spark with 64 or 128 GB. Unsloth budgets about 32 GB of RAM or unified memory for short contexts.
Can Nemotron 3.5 Bolt run on an RTX 4090 or RTX 3090 (24 GB)? #
NVIDIA publishes no 24 GB recipe and we found no measurement. On paper the 22.0 GB GGUF plus the vision file uses about 22.1 GiB of the card's 24 GiB, leaving under 2 GiB for context and buffers (RigCheck arithmetic), so expect to drop the draft model and vision file, keep the context short, or put some layers in system RAM. Unsloth's smaller UD-IQ1_M (20.3 GB) and UD-IQ1_S (18.5 GB) files leave more room, but are not NVIDIA's quantization-aware 2-bit build. The RTX 3090 (Ampere) is also outside the architectures NVIDIA's GGUF card lists (Ada Lovelace, Blackwell).
Can Nemotron 3.5 Bolt run on an RTX 5090? #
Yes, by NVIDIA's "28GB recipe" for discrete RTX GPUs with 32 GB or more: llama.cpp with the GGUF, vision file and draft model on the GPU, 64K context and a q8_0 KV cache. For the full 256K context NVIDIA's recipe asks for a 48 GB GPU.
How much memory does Nemotron 3.5 Super VL need? #
By Unsloth's table, about 64 GB of total memory (RAM plus VRAM, or unified memory) for its 1-bit and smallest 2-bit GGUFs, 80–96 GB for 2- to 3-bit, 96 GB for NVFP4, 128 GB for 4-bit (UD-Q4_K_XL, 86.9 GB) and 192 GB for 8-bit. NVIDIA's official NVFP4 weights (75.0 GB) run on one DGX Spark, one H200 or two H100s; the BF16 weights (248.9 GB) on four H100s.
Can I run Nemotron 3.5 on a Mac? #
Not with an NVIDIA recipe: NVIDIA lists only its own GPUs. Unsloth's llama.cpp guide covers Metal and budgets about 32 GB of unified memory for Bolt; for Super VL its table runs from 64 GB (1-bit) to 192 GB (8-bit), and Ollama lists MLX builds of Super VL (80 GB NVFP4, 130 GB MXFP8). We found no published Mac speed.
Is Nemotron 3.5 on Ollama? #
Yes: ollama run nemotron-3.5-bolt (25 GB) and ollama run nemotron-3.5-super (listed at 80–90 GB), both with image input (Ollama library, checked October 17, 2026).
What is the difference between Nemotron 3.5 Bolt and Super VL? #
Bolt 74B-A9B is compressed from Super VL 120B-A12B (fewer experts, 9 instead of 22 active per token, 256K instead of 1M context) and trained for NVIDIA's 2-bit GGUF so it runs on one desktop machine. Super VL is the full model for DGX Spark and data-center GPUs.
Sources #
- Hugging Face: nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF (llama.cpp recipes, hardware table, file sizes),-24GB (vLLM),-BF16 andLiLiCorr draft model (read October 17, 2026)
- Hugging Face: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 and-NVFP4 (deployment tables, config.json, file sizes)
- NVIDIA blog: Nemotron 3.5 Super VL and Bolt andNVIDIA technical blog: Nemotron 3.5 Bolt 74B-A9B 24GB (runtimes, 24 GB design target, 84 tokens per second on RTX Spark)
- Unsloth guides for Bolt andSuper VL (memory tables) and theBolt andSuper VL GGUF file lists;ggml-org Super VL GGUF
- Ollama library: nemotron-3.5-bolt andnemotron-3.5-super tags;LM Studio: nemotron-3.5-super-vl (checked October 17, 2026)