A new study introduces CaRE, a compute-aware evaluation framework for masked diffusion language models (MDLMs), revealing that current evaluation standards conflate algorithmic improvements with hidde…
A new model compression method called requential coding compresses a generative model by coding a training process built from self-generated data, achieving code length equal to the cumulative teacher…
Researchers introduce token time continuous diffusion (TTCD), a new diffusion language model that operates in continuous space and incorporates per-token times, allowing some tokens to proceed from no…
A 124M-parameter GPT-2 model trained from scratch on OpenWebText data using a custom deep learning library achieved a validation loss of 2.764 nats and a perplexity of 15.87 after 56,000 steps (27.5B …