{"slug": "explorative-modeling-adds-best-of-k-search-to-generative-model-pretraining", "title": "Explorative Modeling adds best-of-K search to generative model pretraining", "summary": "Researchers at the University of Illinois Urbana-Champaign and Harvard University introduced Explorative Modeling (XM), a training method that generates K candidate outputs per example and backpropagates through the lowest-loss candidate, shifting compute into pretraining. Led by NSF Graduate Research Fellow Alexi Gladstone with UIUC professor Heng Ji and Harvard assistant professor Yilun Du, the method reached baseline image-generation results with 6.2 times fewer training samples and 4.1 times fewer total FLOPs, and a large model exploring five candidates outperformed an extra-large baseline with 47% more parameters. The results were published in a July 29th arXiv preprint with PyTorch code released on GitHub.", "body_md": "[Alexi Gladstone (@AlexiGlad)](https://x.com/AlexiGlad) detailed a new generative-model training method on July 31st that searches several candidate outputs for each example and trains on the closest match, shifting compute into pretraining in an effort to produce better models and cheaper inference.\n\nGladstone, an NSF Graduate Research Fellow and computer science PhD student at the University of Illinois Urbana-Champaign, led the [Explorative Modeling](https://explorative-modeling.github.io/) project with UIUC professor [Heng Ji (@hengjinlp)](https://x.com/hengjinlp) and Harvard assistant professor [Yilun Du (@du_yilun)](https://x.com/du_yilun). UIUC's BLENDER Lab lists Gladstone's research areas as world models, system-2 thinking, self-supervised learning and multimodal learning, along with previous internships at Meta and Palantir.\n\nThe researchers argue that exploration can serve as a third pretraining axis alongside model parameters and data. Their evidence comes from experiments spanning image generation, video generation, masked diffusion language models, robot policies and goal-conditioned world models. The results were published in a [July 29th arXiv preprint](https://arxiv.org/abs/2607.27372), with [PyTorch code released on GitHub](https://github.com/alexiglad/XM).\n\n### Moving search into training\n\nGenerative models face a basic problem: a prompt or input can have many valid outputs. A model trained to make one direct prediction under a reconstruction loss can converge on an average of those possibilities, producing a result that matches none of them. Diffusion and autoregressive systems avoid much of that averaging by breaking generation into smaller denoising steps or token predictions.\n\nExplorative Modeling, or XM, takes another route. At each training step, the model generates K candidates, measures each against the target and backpropagates through the lowest-loss candidate. The simplest implementation is a best-of-K loop. In the released repository, researchers can enable the method with an `--xm_best_of_k K`\n\nflag; K=1 is the baseline without exploration.\n\nEach extra candidate carries a compute cost. The paper estimates that an additional candidate in Forward XM costs roughly one-third of a standard transformer training step because it requires another forward pass without another full backward pass. Candidates can also be folded into the batch dimension and processed in parallel. The method therefore makes a direct trade: spend additional compute while training to find a better pairing between latent noise and data, then potentially recover that spending through faster convergence or fewer generation steps.\n\nGladstone wrote in a [project essay](https://alexiglad.github.io/blog/2026/explorative_modeling/) that the work grew from a first-principles attempt to explain why generative modeling is difficult. \"We scale the size of generative models and how much data we train them on... so why haven't we scaled what they can generate?\" he wrote.\n\n### The reported gains\n\nThe authors report that adding exploration to an image-generation recipe reached the baseline's best result with 6.2 times fewer training samples and 4.1 times fewer total FLOPs. A large model exploring five candidates also outperformed an extra-large baseline with 47% more parameters, according to the paper.\n\nThose figures measure the experiments run by the authors, rather than a general reduction that can be assumed for other architectures. The paper reports that exploration's relative gains increased from 7% to 36% as data scaled and from 13% to 23% as model size grew. Image and video metrics improved as K increased across the tested range, while an explorative masked diffusion language model produced a better perplexity-diversity tradeoff than its baseline.\n\nThe inference results may prove more consequential if they reproduce at larger scale. An Explorative Policy matched or exceeded the paper's Diffusion Policy baseline across five robot-manipulation tasks while using one network forward pass instead of 100. On Maze2D planning tasks, the researchers reported a higher average score than a Diffuser baseline while using between 16 and 256 times fewer inference steps.\n\n### The scaling claim still faces a larger test\n\nThe paper's central claim reaches beyond the scale of its current experiments. The image work uses class-conditional ImageNet at 256 by 256 pixels, while the video experiments model 10 frames at 128 by 128 resolution. The researchers say higher-resolution video experiments exceeded their compute budget. Independent replication and tests on larger models will determine whether exploration remains compute-efficient when training costs reach frontier scale.\n\nLanguage is another boundary. The paper reports clearer results for masked diffusion language models and says autoregressive language models were harder to improve because introducing a searchable latent variable is less natural. The authors describe their autoregressive results as modest and leave fuller experiments for later work. That distinction matters because the dominant large language model architecture remains autoregressive.\n\nFully end-to-end image generation also creates a cost problem: the number of candidates may need to grow with the number of modes in a complex distribution. The researchers propose a reverse version of XM that searches data targets for each generation at far lower compute cost, though that approach requires an entropy or coverage constraint to prevent collapse.\n\nExplorative Modeling remains an academic and open-source project affiliated with UIUC and Harvard. [Laude Institute included it in its third Slingshots grant cohort](https://www.laude.org/updates/slingshots-three) in June, and the paper credits support from a Flapping Airplanes fellowship and the National Science Foundation Graduate Research Fellowship Program.\n\nThe near-term value is testability. XM can be added to existing diffusion and flow training pipelines without replacing their underlying architecture, giving research teams a concrete way to measure whether extra candidate search during pretraining buys enough convergence, quality or inference savings to justify its compute bill.", "url": "https://wpnews.pro/news/explorative-modeling-adds-best-of-k-search-to-generative-model-pretraining", "canonical_source": "https://runtimewire.com/article/explorative-modeling-best-of-k-generative-pretraining", "published_at": "2026-07-31 21:25:25+00:00", "updated_at": "2026-07-31 21:27:29.619071+00:00", "lang": "en", "topics": ["machine-learning", "generative-ai", "artificial-intelligence"], "entities": ["Alexi Gladstone", "University of Illinois Urbana-Champaign", "Heng Ji", "Yilun Du", "Harvard University", "Explorative Modeling", "arXiv", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/explorative-modeling-adds-best-of-k-search-to-generative-model-pretraining", "markdown": "https://wpnews.pro/news/explorative-modeling-adds-best-of-k-search-to-generative-model-pretraining.md", "text": "https://wpnews.pro/news/explorative-modeling-adds-best-of-k-search-to-generative-model-pretraining.txt", "jsonld": "https://wpnews.pro/news/explorative-modeling-adds-best-of-k-search-to-generative-model-pretraining.jsonld"}}