Nvidia Nemotron 3.5 Bolt E4B 4GB

Nvidia released Nemotron 3.5 Bolt E4B 4GB, an open small language model built on Gemma 4 E4B with Nemotron post-training and designed to run conversational agents in roughly 4GB of memory on GeForce RTX, Jetson, RTX Spark and DGX Spark hardware. Nvidia reports the model delivers about 40% higher average quality than its prior Nemotron 3 Nano 4B on key conversational agent benchmarks and outputs up to about 80% fewer tokens than peers. Bolt E4B adds multi-turn tool use, a conversational mode for speech-ready responses, configurable reasoning, a 64k context window and support for 35 languages, targeting device assistants, interactive characters and embedded industrial agents.

NVIDIA Nemotron 3.5 Bolt E4B 4GB Published: Technical Report https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-Technical-Report.pdf Introducing NVIDIA Nemotron 3.5 Bolt E4B 4GB Today, we’re excited to release Nemotron 3.5 Bolt E4B 4GB https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB — our latest open small language model optimized for speed and intelligence per GB in conversational agents. Bolt E4B builds on Gemma 4 E4B with Nemotron post-training. It combines natural conversation with multi-turn agentic capabilities to power voice/device assistants, interactive characters and robotic/IoT interfaces running on NVIDIA GeForce RTX https://www.nvidia.com/en-us/geforce/rtx/ , Jetson https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/ , RTX Spark https://www.nvidia.com/en-us/products/rtx-spark/ and DGX Spark https://www.nvidia.com/en-us/products/workstations/dgx-spark/ . Bolt E4B offers ~40% higher average quality against our prior model https://huggingface.co/blog/nvidia/nemotron-3-nano-4b on key conversational agent benchmarks and outputs up to ~80% fewer tokens vs peers. Designed to pack conversational agent intelligence into as little as ~4GB in its target workloads, Bolt E4B enables responsive local experiences while preserving headroom for applications and other models in the same pipeline. Model Features - Multi-turn tool use : call application APIs to carry out tasks across a conversation, with integration guidance for asynchronous tool calls - Conversational mode : toggle to encourage concise, speech-ready responses for natural back-and-forth interactions - Configurable reasoning : enable reasoning for complex requests or disable it for faster responses, with integration options to cap thinking to meet latency targets - 64k context : accommodate conversation history, instructions, tool definitions, retrieved information and application state - Multilingual capability : support for 35 languages, building on a base model pre-trained on +140 languages Agents On Every Device We’re always learning from how NVIDIA Jetson, GeForce RTX and DGX Spark customers — enterprises and enthusiasts alike — use our hardware to accelerate their workloads. As language models become more capable, we’re seeing growing customer interest in using them for experiences like: - Device & Voice Assistants : Fast local assistants for PC, mobile, app, home or OS control; customer service; and in-cabin experiences for vehicles - Virtual & Interactive Characters : In-game NPCs, companions, and other interactive characters that blend dialogue with actions while maintaining a persona - Embedded & Industrial Agents : Human-machine interfaces for robotics and heavy machinery; system automation and industrial IoT Many large models are capable of powering these agents — but constraints around memory, latency, hardware compatibility, cost, or privacy concerns prevent developers from deploying these in production. And so, they turn to faster, cheaper, smaller local models that can unlock these experiences. Evaluating Real-Time Interactive Agents Choosing a model for these experiences is not as simple as picking the highest score on today’s LLM leaderboards. A model that excels at agentic coding is often not the best fit for a fast home assistant that’s fun to interact with — these agents need a different mix of capabilities. Through conversations with customers, several common characteristics emerge: - The model must often embody an agent that converses naturally with a human in the same physical or virtual space - It must know whether, when and how to interleave dialogue with correct tool calls to take actions - It must reliably honor explicit behavioral, persona, brand, or output constraints - It must understand its environment by interpreting supplied app/world telemetry, retrieved database payloads or tool results - And, it must ground all dialogue and actions in the relevant parts of all that context — especially amid distractors or as that context grows throughout a conversation Current benchmarks only partially capture what makes these agents work well, and no individual benchmark covers every capability an application needs. Ultimately, benchmark gains must translate into better behavior in the target application. Model Benchmarks & Performance We selected complementary benchmarks that closely reflect customer workloads or that customers already use, and grouped them into five capabilities. The chart below compares Bolt E4B with models of different sizes on representative benchmark quality, time to first token and VRAM usage: Conversation & Persona | Benchmark metric | Description | Nemotron 3.5 Bolt E4B 4GB | Nemotron 3 Nano 4B | Qwen 3.5 4B | Gemma 4 E4B | |---|---|---|---|---|---| | Brevity Bench avg. quality / conciseness | Answer quality & conciseness | 82.5 | 73.6 | 75.3 | 80.0 | | EQ-Bench v3 https://eqbench.com/eqbench3.html Rubric | Emotional intelligence in roleplay | 39.4 | 19.3 | 32.9 | 41.1 | | MultiChallenge https://labs.scale.com/leaderboard/multichallenge Overall score | Consistency across turns | 53.1 | 35.3 | 51.8 | 42.1 | | RoleBench https://github.com/InteractiveNLP-Team/RoleLLM-public Custom Win % vs. reference | Character speaking style | 67.0 | 12.0 | 53.7 | 49.9 | | RoleMRC https://github.com/LuJunru/RoleMRC Judge avg., excl. multi-turn | Role identity adherence | 80.6 | 47.0 | 65.8 | 74.6 | Multi-turn Tool Use | Benchmark metric | Description | Nemotron 3.5 Bolt E4B 4GB | Nemotron 3 Nano 4B | Qwen 3.5 4B | Gemma 4 E4B | |---|---|---|---|---|---| | BFCL-v3 https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard Avg accuracy | Correct tools & arguments | 68.5 | 61.1 | 70.9 | 66.4 | | Tau2-Bench https://taubench.com/leaderboard/?benchmark=core Avg accuracy | Multi-turn customer tasks | 55.1 | 29.2 | 75.1 | 38.5 | | ToolSandbox https://github.com/apple-aiml-research/ToolSandbox Avg accuracy | Tool use with changing state | 68.8 | 57.9 | 69.5 | 65.4 | | When2Call-MT Overall accuracy | When to call, clarify or abstain | 63.4 | 27.7 | 37.8 | 62.9 | Instruction & Control | Benchmark metric | Description | Nemotron 3.5 Bolt E4B 4GB | Nemotron 3 Nano 4B | Qwen 3.5 4B | Gemma 4 E4B | |---|---|---|---|---|---| | AgentIF https://agentif.github.io/ Instruction success rate | Complex agentic instructions | 25.8 | 20.1 | 20.4 | 21.9 | | IFBench https://artificialanalysis.ai/evaluations/ifbench Prompt loose accuracy | Precise output constraints | 63.8 | 60.1 | 58.9 | 42.8 | | IHEval https://github.com/ytyz1307zzh/IHEval Conflict accuracy | Instruction hierarchy/priority | 84.8 | 51.3 | 60.6 | 67.0 | | InverseIFEval https://arxiv.org/abs/2509.04292 Overall accuracy | Counterintuitive instructions | 46.8 | 35.2 | 31.0 | 42.0 | | StructEval-T https://structeval.github.io/ Average accuracy | Valid structured outputs | 80.4 | 63.5 | 76.0 | 79.3 | Context Robustness | Benchmark metric | Description | Nemotron 3.5 Bolt E4B 4GB | Nemotron 3 Nano 4B | Qwen 3.5 4B | Gemma 4 E4B | |---|---|---|---|---|---| | LongBench v2 https://longbench2.github.io/ 48K, Overall accuracy | Understanding long prompts | 50.5 | 31.4 | 59.9 | 49.3 | | LongMemEval https://github.com/xiaowu0162/LongMemEval Overall accuracy | Recall across conversations | 72.1 | 25.2 | 66.8 | 70.9 | | RULER 64K https://github.com/NVIDIA/RULER Overall accuracy | Context retrieval & reasoning | 87.0 | 91.5 | 95.1 | 86.5 | | SysBench https://github.com/PKU-Baichuan-MLSystemLab/SysBench Instruction success rate | Maintain long prompt rules | 67.9 | 60.9 | 67.3 | 65.8 | State & Action Grounding | Benchmark metric | Description | Nemotron 3.5 Bolt E4B 4GB | Nemotron 3 Nano 4B | Qwen 3.5 4B | Gemma 4 E4B | |---|---|---|---|---|---| | GaRAGe https://github.com/amazon-science/GaRAGe Relevance-aware factuality | Facts from supplied sources | 42.6 | 25.2 | 24.7 | 34.7 | | RAGTruth https://github.com/ParticleMedia/RAGTruth F1 | Avoiding unsupported claims | 77.6 | 71.4 | 75.6 | 77.0 | | RelyToolBench https://github.com/X-LANCE/ToolHallucination Native/real pass % | Tool grounding & hallucination | 63.2 | 41.2 | 43.3 | 61.6 | All results BF16. Every benchmark uses the same reasoning mode across models. Reasoning mode varies by benchmark. Brevity Bench and When2Call-MT address gaps in existing evaluations — to be released at a later date. Across all these benchmarks, Bolt E4B achieves roughly 40% higher average scores than our prior model, Nano 4B, and roughly 10% higher scores than other peers evaluated. These quality gains also come with fewer output tokens — up to 80% fewer in reasoning enabled comparisons. Reducing unnecessary generation lowers decoding work and costs, and can shorten response times. Good responses still need to be fast responses in their target workloads — typically single session batch size 1 envelopes between 2.5K and 16K input tokens and fewer than 500 output tokens. The table below reports peak memory usage, time to first token, and generation throughput across representative hardware and workloads. RTX | SKU | Cache / ISL / OSL | VRAM usage GB | TTFT ms | TPS tokens/s | |---|---|---|---|---| | RTX 5070 12GB | 0 / 3K / 50 | 3.5 | 548 | 133.4 | | | 32k / 2K / 256 | 4.3 | 1,238 | 106.2 | | RTX Spark 32GB | 0 / 3K / 50 | 3.1 | 930 | 67.0 | | | 32k / 2K / 256 | 4.2 | 1,988 | 52.0 | Inference performance across hardware. All configurations measured using Q4 K M at batch size 1 on llama.cpp CUDA with AIPerf. Cache/ISL/OSL denotes cached context, new input, and output lengths in tokens. Results may vary with hardware, inference software, and configuration. Jetson | SKU | Cache / ISL / OSL | VRAM usage GB | TTFT ms | TPS tokens/s | |---|---|---|---|---| | Jetson Orin Nano 8GB | 0 / 3K / 50 | 4.2 | 5,785 | 15.5 | | | 32k / 2K / 256 | 4.9 | 6,259 | 13.0 | | Jetson AGX Orin 32GB | 0 / 3K / 50 | 4.2 | 3,687 | 24.0 | | | 32k / 2K / 256 | 5.0 | 4,091 | 20.3 | Inference performance across hardware. All configurations measured using Q4 K M at batch size 1 on llama.cpp CUDA with AIPerf. Cache/ISL/OSL denotes cached context, new input, and output lengths in tokens. Results may vary with hardware, inference software, and configuration. Training Methodologies & Open Recipes Bolt E4B builds on Gemma4-E4B-it https://huggingface.co/google/gemma-4-E4B-it through Nemotron post-training methods across environments targeting instruction following, multi-turn tool-calling and conversational capabilities. We used reinforcement learning with verifiable rewards RLVR and from human feedback RLHF , powered by NeMo-RL https://github.com/nvidia-nemo/rl and NeMo-Gym https://github.com/NVIDIA-NeMo/gym . The training datasets are listed below, and the full recipe is available here https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/model-coverage/vlm/nvidia/nvidia-nemotron-3-5-bolt-e4b-4gb-bf16.mdx : - Tool-Calling: Conversational Tool Use Pivot https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-Pivot-v1 - Tool-call schema: Function Calling Pivot https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1 - Instruction Following: Instruction Following https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction following , SysBench https://huggingface.co/datasets/nvidia/Nemotron-RL-SysBench-v1 , InverseIFEval https://huggingface.co/datasets/nvidia/Nemotron-RL-InverseIFEval-v1 , CFBench https://huggingface.co/datasets/nvidia/Nemotron-RL-CFBench-v1 , MultiTurnChat https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1 , Calendar https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Calendar-v2 - Instruction Hierarchy: System-User-Instruction-Hierarchy https://huggingface.co/datasets/nvidia/Nemotron-System-User-Instruction-Hierarchy-v1 - Structured Outputs: Structured Outputs https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Structured-Outputs-v2 - Free-form formatting: Free-Form Formatting https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Free-Form-Formatting-v1 - GenRM: Updated version of Nemotron-RLHF-GenRM-v1 https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1 To emphasize non-reasoning accuracy, we used every sample in the RLVR/RLHF training blend in both reasoning and non-reasoning modes. We profiled rewards for each dataset separately in both modes, and maintained a 2:1 ratio of non-reasoning to reasoning samples in each training batch. As post-training progressed, both reasoning traces and final answers became substantially longer. To control verbosity, we increased group-relative length penalties for reasoning and answer length in the GenRM environment. In all other environments, we disabled overlong filtering. After post-training, we quantized the selected BF16 checkpoint to Q4 K M using llama.cpp with importance-matrix imatrix calibration. Calibration used approximately 1M tokens of RL rollouts from the same training step as the selected checkpoint. This release also includes a DFlash parallel drafter. It proposes all draft tokens in one forward pass, which the target verifies in parallel while preserving the target model’s output distribution. DFlash delivers up to 21% higher throughput relative to Gemma-4-E4B’s original MTP drafter. Check out our technical report https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-Technical-Report.pdf for more details Deploy it, Configure it, and Customize it for Your Use Case Get started with Bolt E4B today through Hugging Face https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB , Ollama https://ollama.com/library/nemotron-3.5-bolt , LM Studio, Unsloth https://unsloth.ai/docs/models/nemotron-3.5-bolt , Canonical, Jetson AI Lab https://www.jetson-ai-lab.com/models/Nemotron-3-5-Bolt-E4B-4GB and the NVIDIA In-Game Inferencing SDK https://developer.nvidia.com/rtx/in-game-inferencing . Explore our model card and deployment snippets, training recipes, and prompting/app setup guidelines. We’re excited to see what you build Checkpoints - NVIDIA-Nemotron-3.5-Bolt-E4B-4GB https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB - Post-trained model in GGUF - NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash - Drafter model for the GGUF checkpoint - NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 - Post-trained model in BF16 HF - NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4 https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4 - Drafter model for the HF checkpoint