MDLM

A 115.7M-parameter masked diffusion language model trained with Explorative Modeling (XM) at K=3 for 10,000 steps on one RTX 4090 matched the compute of a 20,000-step baseline but showed a 0.19 nats h…

Researchers propose UNIFUSION, a continual pre-training approach that adapts pretrained autoregressive language models (e.g., GPT-2) to uniform-noise discrete diffusion under a unified reverse-rate ob…