Dory1.0-4B: A Looped Normalized-GPT

A 4-billion-parameter looped (latent-recurrent) language model called Dory1.0-4B was introduced, built on a normalized Transformer (nGPT) backbone with interleaved sliding-window and full-attention layers. Dory1.0-4B offers elastic depth, meaning a single checkpoint serves different inference compute budgets.

We introduce Dory1.0-4B, a 4-billion-parameter looped latent-recurrent language model with elastic depth: a single checkpoint serves different inference compute budgets. Dory1.0-4B is built on a normalized Transformer nGPT backbone with interleaved sliding-window and full-attention layers, organ