Learnability-Informed Fine-Tuning of Diffusion Language Models

Researchers have developed LIFT, a new fine-tuning algorithm for diffusion language models that improves reasoning capabilities by aligning token learning difficulty with the information available at different diffusion time steps. The method outperforms existing supervised fine-tuning baselines across six reasoning benchmarks, achieving up to a threefold relative gain on the AIME'24 and AIME'25 math reasoning tests.

arXiv:2605.22939v1 Announce Type: new Abstract: We aim to improve the reasoning capabilities of diffusion language models DLMs . While SFT is a popular post-training recipe for autoregressive models, its use in DLMs faces challenges and can even hurt performance, though the underlying causes remain understudied. Our analysis reveals that vanilla SFT overlooks learnability, namely what and when tokens are learned. Specifically, rare tokens are difficult to learn when most of the input is masked, whereas it is straightforward and thus of little value to learn common tokens when most of the input is unmasked. Motivated by our analysis, we propose LIFT, an efficient SFT-based post-training algorithm for DLMs. LIFT learns easy tokens when most of the input is masked and hard tokens when more context is available, thus aligning the training with the information available at different diffusion time steps. Our results show that LIFT outperforms existing SFT baselines across six reasoning benchmarks, achieving up to a 3x relative gain on AIME'24 and AIME'25. Our code is publicly available at https://github.com/divelab/LIFT.