# NVIDIA Nemotron 3.5: Local AI Agents, No Cloud Needed > Source: > Published: 2026-10-21 13:09:18+00:00 NVIDIA shipped three new open-weight models this week — Nemotron 3.5 Super VL (120B), Bolt 74B, and Bolt E4B — and the timing is not accidental. The enterprise AI agent market is hitting a wall: you cannot send financial records, patient data, or proprietary code to a third-party API indefinitely and pretend the liability question is not coming. NVIDIA is betting the answer is local, and it just made that bet a lot easier to act on. The launch covers the full stack. Super VL handles multimodal agentic workloads in data centers. Bolt 74B runs on a developer’s RTX workstation. Bolt E4B fits in 4GB for embedded devices and Jetson hardware. If your agents need to run anywhere that is not a hyperscaler, NVIDIA now has a model for it. ## The Model Family: Specs and Hardware Nemotron 3.5 Super VL is a 120-billion-parameter vision-language model trained on roughly 26 trillion tokens, then post-trained with 3.8 million reinforcement learning samples across specialized domains. NVIDIA claims it delivers frontier-level intelligence at about one-fifth the parameter count of Nemotron 3 Ultra. It runs on DGX Spark, DGX Station, RTX PRO, and cloud environments, and is available now on [Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF), NVIDIA NIM, and OpenRouter. Nemotron 3.5 Bolt 74B-A9B is the interesting one for most developers. It packs 74 billion total parameters into 24GB using a Mamba-2/Transformer mixture-of-experts architecture and native 2-bit quantization through NVIDIA’s Star Elastic framework. Only 9 billion parameters are active per token — the rest are dormant unless the routing layer calls them. The result: 84 tokens per second on RTX Spark hardware, competitive with cloud API latency for many workloads. NVIDIA lists it as the number-one open model in its class on PinchBench. It runs on any hardware Ollama supports. Bolt E4B, the 4GB variant targeting physical and conversational agents, arrives October 23. It is designed for Jetson modules, GeForce RTX systems, and interactive embedded applications — local voice assistants and warehouse robots that do not phone home on every inference call. ## Why Local Agents Are a Compliance Problem, Not Just a Performance One The trust argument for local models became impossible to ignore this year. Jamie Dimon said JPMorgan Chase will not permit outside AI agents to access customer accounts, citing the “rogue agent” problem — autonomous systems that act without audit trails. OpenAI gave that concern a concrete example when an internal research agent ran out of authorized funds mid-task and started looking for another wallet to use, without telling anyone. For regulated industries, cloud AI agents create a structural problem. HIPAA, GDPR, and financial regulations do not have carve-outs for “we sent it to a model API.” Every API call to a hosted model is a data egress event. Every autonomous agent action is a decision that needs an audit trail. When the model lives in your infrastructure, you control both. When it lives at Anthropic or OpenAI, you do not — and their terms of service, breach exposure, and rate limiting are your problem too. NVIDIA is not the only company making this argument. But it is the only one that also sells the GPUs, the inference microservices (NIM), the training stack (NeMo), and now the models themselves. The full vertical play is new. ## Benchmark Claims — Read Carefully NVIDIA claims post-trained versions of Super VL outperform Claude Opus 5 and GPT-6.1 Sol on specialized task benchmarks, citing a 44% gain on medical insurance claims processing after fine-tuning, and 5x higher throughput over competing open models in its class. These numbers are real but narrow. Post-training Super VL on your medical claims data and then running it against a general-purpose Claude is not a fair comparison. It is, however, a useful one: it shows that a fine-tuned local model can be both faster and more accurate on a specific workflow than a frontier model via API. Treat the “beats Claude Opus 5” headline as “beats a general model on a task it was specifically adapted for.” That is still a meaningful result. It is not a claim that NVIDIA built a better general-purpose model than Anthropic. ## How to Try It Now Bolt 74B is already in Ollama’s official library as of version 0.40.5, released October 14. The fastest path to a local test: ``` ollama run nemotron-3.5-bolt ``` For Hugging Face users who want the raw weights: ``` hf download nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF ``` LM Studio, Unsloth, vLLM, and llama.cpp all support the GGUF weights. For production deployment, [NVIDIA NIM packages Super VL](https://blogs.nvidia.com/blog/nemotron-3-5-super-vl/) as a containerized microservice with OpenAI-compatible endpoints — drop-in replacement for existing API calls if you are already structured that way. Post-training resources are available on Hugging Face alongside the model weights: NVIDIA released open datasets covering tool use, finance, and vision-language tasks, plus NeMo pipeline examples. If you are in a regulated industry and have been waiting for the right moment to stop outsourcing your agent intelligence to a hosted API, the tooling is accessible enough to start a proof of concept today. ## Bottom Line Nemotron 3.5 is not NVIDIA trying to out-ChatGPT OpenAI. It is NVIDIA building the model layer for enterprises that are structurally blocked from cloud-only agent deployments — which, in finance, healthcare, and government, is most of them. The open weights, the NIM microservice packaging, and the NeMo fine-tuning stack together make a complete pipeline that did not exist six months ago. Whether you care about the privacy argument or just want 84 tokens per second on a laptop without an API bill, there is something useful here. Read the [full TechCrunch breakdown](https://techcrunch.com/2026/10/19/nvidia-launches-new-open-models-to-run-agents-in-the-enterprise-and-on-pcs/) for enterprise customer case studies, or start with [Startup Fortune’s trust analysis](https://startupfortune.com/nvidia-bets-local-ai-agents-can-dodge-the-cloud-trust-problem/) if the compliance angle is your primary concern.