Reactive Optimizers: rewrite the optimizer's source code during training A researcher Xingjian Bai released an open-source OPAL package implementing "reactive optimizers" that place an agent inside the training loop to watch training dynamics and rewrite the optimizer's source code mid-run, generating the weight update rule for each parameter group. The zero-shot approach, which retains nothing from repeated runs, matched or beat the best off-the-shelf optimization recipe across image and language tasks when no recipe had been tuned for the task, and spontaneously produced schedules and strategies that suppressed loss spikes and escaped overfitting. Bai positions the method as most useful where no recipe exists yet, such as a new architecture or dataset, and expects it to strengthen with each generation of agents. ← Xingjian Bai https://xingjianbai.com/ Reactive Optimizers TL;DR We put an agent at the heart of the optimizer. It watches the training dynamics, reacts to them, and rewrites the optimizer’s source code as training goes. When no recipe has been tuned for a task, it matches or beats the best recipe one could pick off the shelf, across image and language tasks. Further, we demonstrate that it spontaneously comes up with sophisticated schedules and intricate strategies to prevent loss spikes and escape overfitting. Try it on your own training run with the OPAL package https://github.com/xingjian-bai/opal . Today, the recipe used to optimize a deep neural network is typically completely determined before training even begins. A great deal of human ingenuity has gone into priors regarding how training should progress: adaptive step sizes 1 r-adam , preconditioning 2 r-shampoo , 3 r-soap , orthogonalized updates 4 r-muon , 5 r-moon , schedules tied to the training step 6 r-sgdr , 7 r-wsd , and learning rates scaled with model size 8 r-mup . Automated discovery 9 r-ais , 10 r-pi , 11 r-lion , 12 r-velo searches across many runs for better optimizers. Whether hand-crafted or automatically discovered, the optimizer then runs unchanged through the end of training, adapting, if at all, according to rules written in advance. This scope leaves the following questions unanswered: How should an optimizer respond to an unexpected loss spike or stall? What if one module's gradients start to oscillate while the others keep improving smoothly? And when a new type of architecture trains poorly with Adam or Muon, is the architecture worse, or has it not yet met an optimizer suited to it? In this post, we attempt to expand the scope of conventional optimizers and make them reactive to the training itself, leading to an ever-adapting optimizer inside a single training run. This reactive optimizer reasons on the spot and writes the weight update rule for each parameter group. Directly writing optimization code, it has the freedom to implement complicated schedules, respond to training accidents, and customize the optimizer for the single training episode in which it exists. Building on earlier attempts to adapt optimizers during training 12 r-velo , 13 r-daniel , 14 r-pbt and to let a model direct its own fine-tuning 15 r-seal , the reactive optimizer is entirely zero-shot, with nothing learned or retained from repeated runs, and puts the optimizer's entire algorithm up for rewriting. We start with a practical situation, with tasks from image generation and language modeling, in which a reactive optimizer can do much better than static ones. We then turn to the emergent properties of the reactive optimizer, including sophisticated scheduling, suppressing loss spikes, and escaping overfitting. Lastly, we ask what all this costs and how practical it is at scale. We expect the reactive optimizer to grow only stronger with each generation of agents, to be most useful exactly where no recipe exists yet for a new architecture or a new dataset , and to be one step toward solving the puzzles of deep learning optimization. 1. Training without a recipe Suppose a PhD student has a new idea: a model of their own design, or a new dataset on which to train a transformer. They have one GPU for a single day only and a modest allotment of LLM API tokens, allowing for just one serious training run. What should the student do? The obvious course of action would be to pick the most standard training recipe, or one from a recent paper, and cope with the possibility that it might be unsuitable for their scenario, with spiking/stalling loss, etc. The underlying question is broader: when bounded by GPU time, researcher time, or wall-clock time, how can a deep learning model be trained to its best performance without knowing the optimal optimization recipe in advance? New architectures, modalities, languages, and scientific data all fall into this setting, where few people can afford a sweep at scale. A reactive optimizer offers another way to use that budget: it spends LLM tokens in place of GPU/human hours. With optimizers reacting to training incidents and adjusting themselves, much of the burden of recipe tuning can be reduced. Results in brief We start from the optimization track Track 3 of modded-nanogpt \ 16\ r-mng , a language-model pre-training task training a 162M GPT on FineWeb with many recipes tuned specifically for this one setting. From this task, we create two slightly differing natural settings: 1 the same model trained on an alternate language dataset we select Chinese, Thai, and Kyrgyz , and 2 the original setup with a quarter of the batch size. Neither comes with a recipe tuned for that setting, so we compare the reactive optimizer with the natural options one would have if they wanted to one-shot the new training: adopting five leaderboard recipes with natural adjustments. In all these new settings, the reactive optimizer ranks first among all baselines Figure 1 fig-reel . We also apply the reactive optimizer directly to Goldfish 17 r-goldfish , which proposes a dataset of 350 languages and trains one small GPT per dataset with a standard recipe not tuned for each language. This makes it an ideal ground to test the optimizer's adaptiveness in one shot. On the Amharic dataset, the official recipe ends at 83.48 bits per sentence on held-out FLORES-200 text lower is better ; the reactive optimizer, initialized with this same recipe, ends at 81.11 in one shot on Swahili, we obtain a similar gain . Sweeping the learning rate of the standard recipe finds a static recipe that ends ahead of the reactive optimizer, at 80.89, but the reactive optimizer initialized from that swept recipe ends ahead again, at 80.73. We also try the reactive optimizer on image generation, a direction where the literature usually does not sweep optimization hyperparameters, since diffusion-model design and optimizer design are viewed as mostly orthogonal