{"slug": "announcing-day-0-support-for-nvidia-nemotron-3-5-super-vl-on-vllm", "title": "Announcing Day 0 Support for NVIDIA Nemotron 3.5 Super VL on vLLM", "summary": "VLLM shipped Day 0 support for NVIDIA Nemotron 3.5 Super VL, an open multimodal model built on a 120-billion-parameter mixture-of-experts architecture with 12 billion active parameters, in vLLM release v0.31.0. The model accepts text, images, documents and video as input and returns text, and is positioned as the high-accuracy agentic execution model alongside Nemotron 3.5 Lightning for high-volume text and Nemotron 3 Ultra for text-only reasoning. vLLM serves it with continuous batching, prefix caching, speculative decoding and an OpenAI-compatible API, with BF16 and NVFP4 checkpoints available at launch and deployment targets including 1x GB10, B200, B300, RTX PRO 6000, H200 and 2x H100.", "body_md": "# Announcing Day 0 Support for NVIDIA Nemotron 3.5 Super VL on vLLM\n\nAlways-on agents increasingly need to interpret visual information, reason across it, use tools, and act. These workflows require a model that combines multimodal understanding with high-accuracy execution.\n\nNVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal model built for long-running agents, enabling developers to maximize accuracy and efficiency for specialized workflows.\n\nIt combines reasoning, instruction following, and tool use with text, image, document, and video understanding in one model, reducing the need to deploy separate text and vision models and helping maintain task-relevant context across workflow steps.\n\nBuilt on a 120-billion-parameter mixture-of-experts (MoE) architecture with 12 billion active parameters, Nemotron 3.5 Super VL balances accuracy and efficiency for specialized, always-on agent tasks. Organizations can post-train it with NVIDIA NeMo using their data, tools, policies, and workflows.\n\nWithin a system of models, Nemotron 3.5 Super VL is designed as the default high-accuracy agentic execution model with native document, image, and video understanding. It complements NVIDIA Nemotron 3.5 Lightning for high-volume text execution and NVIDIA Nemotron 3 Ultra for the hardest text-only reasoning.\n\nNemotron 3.5 Super VL addresses two practical requirements for multimodal, always-on agents:\n\n- **Native visual understanding with agentic execution:** The model combines high-accuracy reasoning and tool use with document, image, and video understanding, preserving task-relevant context across specialized workflows.\n- **Efficient, controllable deployment:** A hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization are designed to improve generation efficiency and create headroom for more concurrent long-context sessions.\n\n## TL;DR: About NVIDIA Nemotron 3.5 Super VL\n\n- **Architecture:** 120B total parameters with 12B active parameters in a mixture-of-experts design\n- **Modalities:** Text, images, documents, and video as input; text as output\n- **Optimization:** C-RADIOv4 vision encoder, hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization\n- **Model role:** High-accuracy agentic execution with native visual understanding for systems of models\n- **Customization:** Post-train with NVIDIA NeMo using organization-specific data, tools, policies, and workflows\n- **Deployment:** NVIDIA NIM for optimized production serving across cloud and data center environments\n- **Deployment targets:** 1x GB10, B200, B300, RTX PRO 6000, H200, and 2x H100\n- **Availability:**\n\n## Run High-Throughput Inference with vLLM\n\nNemotron 3.5 Super VL is intended to run across a wide range of NVIDIA platforms. vLLM provides the serving layer needed to bring the model into production workflows, including continuous batching, prefix caching, speculative decoding, and an OpenAI-compatible API.\n\nThe BF16 checkpoint offers a straightforward baseline for deployment. NVFP4 is also available at launch for environments that can take advantage of lower-precision inference.\n\n### Install vLLM\n\n```\ndocker pull vllm/vllm-openai:v0.31.0\n\ndocker run --rm -it --gpus all --ipc=host --network=host \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  --entrypoint /bin/bash \\\n  vllm/vllm-openai:v0.31.0\n```\n\n### Serve the Model\n\nThe following NVFP4 launch configuration is an example for a 1× GB200 setup. Validate it on the target hardware before use.\n\n```\nvllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \\\n  --served-model-name nemotron-3.5-super \\\n  --trust-remote-code \\\n  --max-model-len 114688 \\\n  --reasoning-parser nemotron_v3 \\\n  --speculative-config.method mtp \\\n  --speculative-config.num_speculative_tokens 5\n```\n\nOnce the server is running, applications can send prompts through an OpenAI-compatible client:\n\n``` python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"null\")\n\nresponse = client.chat.completions.create(\n    model=\"nemotron-3.5-super\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe what happens in this video.\"},\n                {\n                    \"type\": \"video_url\",\n                    \"video_url\": {\"url\": \"https://example.com/clip.mp4\"}\n                },\n            ],\n        }\n    ],\n    temperature=1.0,\n    top_p=0.95,\n    max_tokens=4096,\n)\n\nprint(response.choices[0].message.reasoning)  # thinking trace\nprint(response.choices[0].message.content)    # final answer\n```\n\n## Accelerate Long-Running Agentic Workflows with Speculative Decoding\n\nNemotron 3.5 Super VL supports Multi-Token Prediction (MTP) to accelerate token generation while preserving the target model’s output quality.\n\nNemotron 3.5 Super VL includes built-in multi-token prediction (MTP) heads. During decoding, these heads propose future tokens and the target model verifies them, reducing the number of sequential generation steps required for longer responses.\n\nFor example, MTP can also be enabled with three speculative tokens:\n\n```\nvllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \\\n  --served-model-name nemotron-3.5-super \\\n  --trust-remote-code \\\n  --speculative-config.method mtp \\\n  --speculative-config.num_speculative_tokens 3\n```\n\n## Built for Specialized, Always-On Agent Workflows\n\nOrganizations can adapt Nemotron 3.5 Super VL to their terminology, data, policies, tools, and workflows with NVIDIA NeMo. This enables a general multimodal model to become a specialized execution engine for domain-specific agent tasks while preserving control over model behavior and data handling.\n\n### Example Use Cases\n\n- **Customer service:** Resolve multi-step requests by interpreting customer-provided documents and images, applying policy, and using service tools.\n- **Document intelligence:** Interpret complex documents - including OCR, tables, charts, and page layouts - to extract information and generate summaries or reports.\n- **Enterprise productivity:** Complete multi-step tasks by interpreting visual information, reasoning across it, and using tools.\n- **Video understanding:** Extract key information from video to support search, content review, advertising, retail, and incident analysis.\n- **Web research:** Search across websites, connect information from multiple sources, and verify results.\n\n## Accuracy and Efficiency for Multimodal Agent Tasks\n\nNemotron 3.5 Super VL is designed to deliver high accuracy for specialized multimodal agent tasks while using only a subset of its total parameters for each token. The hybrid Mamba-Transformer architecture and multi-token prediction along with NVFP4 quantization help accelerate output generation.\n\nAs shown in Figure 1, on a single GPU, this model delivers up to 5x faster throughput for agentic tasks compared to other [open models](https://www.nvidia.com/en-us/glossary/open-models/) in its size category.\n\nFigure 1. Nemotron 3.5 Super VL delivers over 600 TPS on Artificial Analysis Intelligence Index agentic tasks on Baseten endpoint.\n\nAs shown in Figure 2, Nemotron 3.5 Super VL performs tasks 4x faster compared to other leading open models of similar size.\n\nFigure 2. Nemotron 3.5 Super VL takes advantage of higher throughput and token efficiency to deliver 4x faster task completion.\n\nFor context, Figure 3 shows a DGX Spark comparison for the related Nemotron 3.5 Super model. The chart does not measure Nemotron 3.5 Super VL.\n\nFigure 3. Nemotron 3.5 Super on the DGX Spark efficiency frontier.\n\n### Get Started\n\nReady to build agents that can see, reason, and act?\n\n- Download the model weights from Hugging Face: [BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) and[NVFP4](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4)\n- Run Nemotron 3.5 Super VL with vLLM using the getting-started [cookbook](https://github.com/NVIDIA-NeMo/Nemotron/blob/main/usage-cookbook/Nemotron-3.5-Super-VL/vllm_cookbook.ipynb)\n\n### Acknowledgements\n\nNVIDIA: Nirmal Kumar Juluru, Anusha Pant, Faradawn Yang, Pen Chung Li, Alex Steiner, Ran Zilberstein, Nave Assaf, Amir Klein", "url": "https://wpnews.pro/news/announcing-day-0-support-for-nvidia-nemotron-3-5-super-vl-on-vllm", "canonical_source": "https://vllm-project.github.io/2026/10/15/nemotron-3-5-super-vl-vllm.html", "published_at": "2026-10-15 00:00:00+00:00", "updated_at": "2026-10-19 10:45:20.126602+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-agents", "ai-tools", "machine-learning"], "entities": ["NVIDIA", "Nemotron 3.5 Super VL", "vLLM", "NVIDIA NeMo", "NVIDIA NIM", "Nemotron 3.5 Lightning", "Nemotron 3 Ultra", "NVIDIA GB200"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/announcing-day-0-support-for-nvidia-nemotron-3-5-super-vl-on-vllm", "markdown": "https://wpnews.pro/news/announcing-day-0-support-for-nvidia-nemotron-3-5-super-vl-on-vllm.md", "text": "https://wpnews.pro/news/announcing-day-0-support-for-nvidia-nemotron-3-5-super-vl-on-vllm.txt", "jsonld": "https://wpnews.pro/news/announcing-day-0-support-for-nvidia-nemotron-3-5-super-vl-on-vllm.jsonld"}}