BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

A method called Build Your Own DLM (BYOD) converts existing autoregressive language models into diffusion language models using LoRA-only fine-tuning, avoiding the costly pretraining or full-model fine-tuning that leading diffusion systems rely on. BYOD's authors present the recipe as generalizable, targeting the parallel token updates, flexible generation and bidirectional attention that diffusion language models allow but autoregressive models do not.

Unlike autoregressive AR models, diffusion language models DLMs permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM BYOD , a generalizable LoRA-only recipe for pa