Run DiffusionGemma on NVIDIA for Developer-Ready, High-Throughput Text Generation

Developers building real-time AI—such as chat assistants, copilots, and agentic workflows—are often constrained by token-by-token generation speed. This limits responsiveness, increases serving costs, and makes fluid, interactive experiences difficult to achieve.

DiffusionGemma, created by Google DeepMind and optimized to run efficiently across NVIDIA platforms, introduces a new approach to text generation, producing tokens in parallel rather than one at a time, enabling faster, higher-throughput AI applications. The model uses diffusion-based denoising to generate 256 tokens in parallel per step, delivering up to 1,000 tokens/sec on a single NVIDIA H100 Tensor Core GPU, up to 150 tokens/sec on NVIDIA DGX Spark, and up to 2,000 tokens/sec on NVIDIA DGX Station.

For enterprise developers, this speed translates into lower serving costs, higher concurrency, and more responsive user experiences without sacrificing model quality. DiffusionGemma is built on the Gemma 4 26B A4B MoE architecture and optimized for low-latency, memory-bound inference.

Model name | DiffusionGemma | | Supported modalities | Text, image | | Total parameters | 25.2B | | Active parameters | 3.8B | | Context length | Up to 256K tokens | | Precision format | BF16, NVFP4 |

Table 1. Overview of the DiffusionGemma, summarizing modalities, parameter sizes, and supported context length

In addition to NVIDIA data center GPUs, developers can enjoy optimal performance on a variety of client GPUs and systems.

NVIDIA DGX StationDGX Station playbooks;vLLM on DGX Station guideNVIDIA RTX + NVIDIA RTX PRORTX blog;vLLM on RTX guideTable 2.

Comparison of local deployment options across NVIDIA platforms, highlighting primary use cases, key capabilities, and recommended getting‑

started resources for DGX Spark, DGX Station, and RTX + RTX PRO systems

Build and prototype on NVIDIA

Access DiffusionGemma through Hugging Face Transformers for initial testing and prototyping on NVIDIA GeForce RTX 5090 or DGX Spark. For higher throughput or concurrent multi-user serving on DGX Spark, DGX Station, and RTX PRO, use vLLM by following our playbooks in Table 2.

With Day 0 support across NVIDIA hardware and software—from local prototyping to production deployment—developers can quickly move from experimentation to real-world applications.

NVIDIA GPU-accelerated endpoints

Start building with DiffusionGemma with free access for prototyping to GPU-accelerated endpoints on build.nvidia.com as part of the NVIDIA Developer Program. The browser experience can also be connected to custom data sources.

BF16 and NVFP4

The model is available today on Hugging Face with BF16 checkpoints, and an NVFP4 quantized checkpoint for DiffusionGemma is also available using NVIDIA Model Optimizer.

Enterprise deployments with NVIDIA NIM #

NVIDIA NIM makes it simple to deploy DiffusionGemma from development into production. NIM packages the model as an optimized, containerized inference microservice — with performance tuning, standardized APIs, and the flexibility to run on-premises, in the cloud, or across hybrid environments. NIM exposes a standard OpenAI-compatible API for sending inference requests to the server.

• Download the container. - Start the NIM server.

$ export NIM_IMAGE_PATH = “nvcr.io/nim/google/diffusiongemma-26b-a4b-it:latest”
$ docker run --gpus=all \ 
  -e NGC_API_KEY=$NGC_API_KEY \ 
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \ 
  -p 8000:8000 \ 
 ${NIM_IMAGE_PATH}

• Make a test request and read the full NIM documentation.

from openai import OpenAI 
client = OpenAI( 
    base_url="http://localhost:8000/v1", 
    api_key="not-required" 
) 
response = client.chat.completions.create( 
    model="google/diffusiongemma-26b-a4b-it”,
    messages=[ 
        {"role": "user", "content": "Write a poem about text diffusion"} 
    ], 
    max_tokens=256 
) 
print(response.choices[0].message.content)

Day 0 finetune with NVIDIA NeMo AutoModel #

Fine-tuning guides and recipes are available through the NVIDIA NeMo AutoModel library, part of the NVIDIA NeMo Framework, for developers looking to adapt the model to specific tasks or domains. NeMo AutoModel enables users to fine-tune models (LLMs, VLMs and DiffusionLMs) directly on top of HuggingFace checkpoints without conversion, so users can start rapid experimentation on the latest frontier models.

NVIDIA is an active contributor to the open-source ecosystem and has released several hundred projects under open-source licenses. NVIDIA is committed to open models such as DiffusionGemma that promote AI transparency and enable users to share their work in AI safety and resilience.

Check out DiffusionGemma on Hugging Face or test for free using NVIDIA APIs at build.nvidia.com.