DiffusionGemma Grammar and Word-Merging Issues in Question Generation

A user running RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic via vLLM in Docker reports word-merging and grammar issues in a marine-domain question-generation application, such as 'upcomingbunkering operation' instead of 'upcoming bunkering operation', while the same model and server perform well in a RAG chatbot. The user seeks to determine whether the problem stems from the diffusion model's limitations or from configuration differences like canvas_length=256 and diffusion_entropy_bound=0.1.

Hi everyone, I am currently running RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic using vLLM through Docker. My current vLLM command is: vllm serve --model RedHatAI/diffusiongemma-26B-A4B-it-FP8-dynamic \ --trust-remote-code \ --attention-backend TRITON ATTN \ --max-num-seqs 4 \ --max-model-len 8192 \ --gpu-memory-utilization 0.78 \ --generation-config vllm \ --hf-overrides '{"diffusion sampler":"entropy bound","diffusion entropy bound":0.1}' \ --diffusion-config '{"canvas length":256}' \ --host 0.0.0.0 \ --port 8085 I am using the same model/server in two different applications: Interestingly, the model works reasonably well in the RAG chatbot. I don’t see noticeable grammar problems or word-merging issues there. However, in the marine-domain question-generation application , I sometimes get outputs with: For example, I can get output similar to: upcomingbunkering operation instead of: upcoming bunkering operation I also see cases where the generated question structure or grammar is not correct. The important part is that the same model and vLLM server are being used , but the problem is much more noticeable in the question-generation workflow. I understand that DiffusionGemma is experimental and that its output quality may not be comparable to standard autoregressive Gemma models. However, I am trying to understand whether the behavior I’m seeing is actually a model limitation or whether there is something different in my question-generation pipeline that I am missing. Could the difference be caused by things such as: canvas length=256 entropy bound / diffusion entropy bound Has anyone experienced similar word-merging or grammar issues with DiffusionGemma specifically during question generation or structured output generation ? Any suggestions on what I should compare or test between the RAG and question-generation pipelines would be appreciated. I would particularly like to know whether this is expected behavior from the diffusion generation approach or whether there is a configuration issue I should investigate.