Representation-Space MMD for Diffusion Language Models

Researchers introduced a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. The method estimates MMD by retaining contextual features at individual token positions.

We introduce a post-training method for diffusion language models DLMs that minimizes Maximum Mean Discrepancy MMD between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual token positions, obtainin