How to Run Clover 35B-A3B, DiffuSpace's Diffusion Language Model, Locally

DiffuSpace released Clover-35B-A3B-Preview, a block diffusion language model built on Qwen3.6-35B-A3B that generates text in parallel blocks of 32 tokens rather than one token at a time, with 35 billion total parameters and about 3 billion active per token. Running it locally requires roughly 69 GB of GPU memory for BF16 weights alone, with one tester reporting just under 68 GB total consumption, and official setup uses Transformers 5.19.0 with PyTorch 2.10+ and trust_remote_code=True, while DiffuSpace recommends SGLang over plain Transformers for serving because it offers self-speculative decoding and faster inference. DiffuSpace's benchmark charts show Clover leading open diffusion models LLaDA, Dream and Mercury on six agentic benchmarks, though independent testing found real-world gaps including failed coding and reasoning tasks.

How to Run Clover 35B-A3B, DiffuSpace's Diffusion Language Model, Locally A practical guide to installing DiffuSpace's Clover 35B-A3B diffusion model locally, covering VRAM needs, setup steps, and real-world performance. What is Clover 35B-A3B? Clover-35B-A3B-Preview is the first model released by DiffuSpace, and it’s a diffusion language model rather than the usual token-by-token autoregressive design. Instead of generating one word after another, it fills in a whole block of text at once and then moves to the next block. It’s built on top of Qwen3.6-35B-A3B, keeping that model’s mixture-of-experts backbone 40 layers, 256 routed experts, top-8 routing, one shared expert per layer , but adds new learned parameters that let tokens inside a block attend to each other bidirectionally while still preserving a causal prefix. Total parameter count is 35 billion, with about 3 billion active per token, and it supports a 128K token context window for prompt plus response combined. TL;DR - Clover is a block diffusion model built on Qwen3.6-35B-A3B, generating chunks of text in parallel instead of one token at a time, using a default block size of 32 tokens. - VRAM needs sit around 69 GB for the BF16 weights alone, with one tester reporting roughly 68 GB total consumption when serving locally, so a single high-memory GPU or multi-GPU setup is required. - Official setup goes through Transformers 5.19.0 and PyTorch 2.10+ , with trust remote code=True required since Clover ships custom modeling code rather than standard Transformers classes. - DiffuSpace recommends SGLang for serving and evaluation , not the plain Hugging Face Transformers path, because it offers self-speculative decoding and faster inference than the standalone PyTorch implementation. - Benchmark charts show Clover leading other open diffusion models LLaDA, Dream, Mercury on six agentic benchmarks, but independent testing found real-world gaps, including failed coding and reasoning tasks. - Generation includes built-in repetition and length controls presence penalty, repetition penalty, length and thinking-end biases that are already tuned by default, so most users won’t need to touch them. - Expert routing can be restricted per block via a coreset setting that trims which experts get used during denoising, trading some quality for efficiency without shrinking the actual model weights. Remy is new. The platform isn't. Remy is the latest expression of years of platform work. Not a hastily wrapped LLM. How does block diffusion decoding actually work? Clover generates text in blocks instead of single tokens. Each block starts fully masked, and the model resolves the positions it’s most confident about first, filling them in parallel rather than sequentially. If no position clears the confidence threshold, the system still fills in the single most confident position so generation keeps moving forward. Once a block is resolved, it becomes part of the prefix context for the next block. The Hugging Face reference implementation defaults to block size=32 and threshold=0.95 . These aren’t fixed: they’re generation-time keyword arguments you can tune, with higher thresholds generally producing more conservative, resolved-in-fewer-passes behavior and lower thresholds trading accuracy for speed. The tradeoff is architectural rather than cosmetic: bidirectional attention inside a block requires additional learned components right-convolution weights, softmax scalars layered onto the underlying Gated DeltaNet / full-attention MoE backbone inherited from Qwen. What hardware do you need to run it? The BF16 weights alone need approximately 69 GB of GPU memory, and that’s before accounting for inference state like KV cache and activation buffers. In practice, hands-on testing showed VRAM consumption hovering just under 68 GB when serving the model with a custom Transformers script, which lines up with the documented figure. That effectively rules out single consumer GPUs. You’re looking at a workstation-class card like an H100 or A100 80GB , multiple smaller GPUs split via device map , or a rented cloud GPU instance sized for at least 80 GB of VRAM to leave headroom for context and generation state. Given the 128K context ceiling, anyone planning to use long prompts or long outputs should budget extra memory beyond the base 69 GB, since the model card notes that requests exceeding the context limit, including space needed to resolve the final block, are rejected outright. How do you set it up with Transformers? The documented quick start uses Transformers 5.19.0 with PyTorch 2.10 or later. Install the dependencies: pip install "transformers==5.19.0" accelerate safetensors Then load the model with trust remote code=True , since Clover relies on custom modeling and configuration files modeling clover moe.py , configuration clover moe.py , sampling clover moe.py rather than a stock Transformers architecture: python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model id = "Diffuspace/Clover-35B-A3B-Preview" tokenizer = AutoTokenizer.from pretrained model id, trust remote code=True model = AutoModelForCausalLM.from pretrained model id, trust remote code=True, dtype=torch.bfloat16, device map="cuda:0", .eval Prompts go through the standard chat template with enable thinking=True if you want the model’s reasoning trace exposed, and generation accepts the usual max new tokens alongside diffusion-specific arguments like block size and threshold . You can also point model id at a local directory once you’ve downloaded the weights, which matters given the model ships as 26 sharded safetensors files. Is SGLang a better option than Transformers? DiffuSpace’s own documentation recommends the SGLang implementation for serving and evaluation rather than the plain Hugging Face path. The standalone PyTorch implementation in Transformers works and is useful for experimentation, but the model card explicitly flags that small numerical differences between its operations and SGLang’s kernels can produce different generated text, even with identical decoding settings. SGLang also supports self-speculative decoding, which should speed up block resolution compared to the reference PyTorch loop. If you’re setting this up for anything beyond a quick local test, the SGLang path detailed in DiffuSpace’s GitHub repository is the one to use. A llama.cpp implementation is also referenced for self-speculative decoding, though training code itself has not been released. What generation controls does Clover expose? Clover ships with several decoding controls already tuned by default, aimed at preventing long generations from becoming repetitive, a known failure mode in long-context models. These include a presence penalty 0.9 that discourages reusing tokens already in the response, a repetition penalty 1.05 , and two gradual biases: one nudging the model toward ending its response dllm length penalty and one nudging it toward closing its thinking phase dllm think end bias , each activating after a window of several thousand tokens. These are soft nudges, not hard cutoffs; max new tokens remains the actual ceiling. There’s also a block-level expert coreset setting moe block coreset top p , default 0.9 that restricts routing to a subset of experts covering 90% of the aggregated routing mass for a block, rather than letting every token in the block route independently across all 256 experts. This can reduce compute overhead during denoising, though DiffuSpace notes it doesn’t shrink the stored model weights and can carry a quality tradeoff at lower values. Is Clover actually worth running right now? This is where the picture gets more mixed. DiffuSpace’s own benchmark charts show Clover leading other open diffusion models LLaDA, Dream, Mercury across six agentic benchmarks, with the largest gaps on terminal-based and SWE-bench-style coding tasks, and claim it matches or beats its Qwen base model on knowledge, logic puzzles, instruction following, and tool use. Hands-on testing told a different story in several areas. In one real-world debugging test against a full-stack Docker application with a specific inventory-tracking bug, the model reported success but the bug remained unfixed after reload. A single-file HTML canvas animation request produced working but visually rough output, with flat rendering and missing effects. A multilingual naming test across 80-plus languages stalled partway through, produced duplicate outputs across distinct languages, and in some cases mixed scripts within a single language’s answer. A Zebra logic puzzle, the kind of classic reasoning benchmark that should be well represented in training data, came back only half-correct after the model’s internal reasoning trace ran in circles for over 50 minutes before answering. The gap between chart performance and task performance is worth weighing carefully if you’re deciding whether to invest GPU time in this model versus a mature autoregressive alternative. As a first-generation preview release from a new lab, Clover is a legitimate technical experiment in bringing diffusion decoding to a large MoE backbone, but it’s not yet a drop-in replacement for standard coding or reasoning workloads. Frequently Asked Questions How much VRAM does Clover 35B-A3B need to run locally? Plan for at least 69 GB just for the BF16 weights, with real-world usage observed around 68 GB during active serving. Add more headroom for KV cache and context if you’re using long prompts, since the model supports up to 128K tokens total. Can I run Clover on a single consumer GPU? Not practically. Consumer cards with 24 GB or even 48 GB of VRAM fall well short of the roughly 69 GB needed for the BF16 weights alone. You’ll need a datacenter-class GPU like an 80 GB A100 or H100, a multi-GPU setup, or a rented cloud instance sized accordingly. Does Clover support quantized formats to reduce memory use? The official release and quick-start instructions describe BF16 weights served through Transformers or SGLang. Community quantizations may appear over time, but they aren’t part of the documented official release referenced here. Is Clover better than its base model, Qwen3.6-35B-A3B? DiffuSpace’s benchmark charts suggest it holds its own against the base autoregressive model on several categories and loses mainly on the hardest math and exam-style questions. Independent hands-on testing found weaker results in coding, multilingual generation, and logic puzzle accuracy, suggesting the benchmark gains don’t uniformly translate to practical tasks yet. Should I use Transformers or SGLang to serve Clover? DiffuSpace recommends SGLang for serving and evaluation, citing kernel-level numerical differences that can shift output text compared to the Transformers implementation, plus support for self-speculative decoding. Transformers remains useful for quick local experimentation and development.