Researchers report that the staged pipeline used to train foundation models — pretraining, midtraining, supervised fine-tuning (SFT) and reinforcement learning (RL) — is not algorithmically fundamental, and that suitable RL objectives can be derived as maximum-likelihood objectives so a single algorithm covers the whole process. The work argues the separation between these stages, which also involves changes in training data, can be collapsed into one method.
Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms. We show that this algorithmic separation is not fundamental. By deriving suitable RL objectives as maximum-li
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.