{"slug": "gpu-forecasters-language-models-as-selective-surrogates-for-kernel-optimization", "title": "GPU Forecasters: Language Models as Selective Surrogates for Kernel Optimization", "summary": "Researchers have demonstrated that large language models can serve as selective surrogates for GPU kernel performance evaluation, accurately forecasting kernel runtime without requiring repeated compilation and execution on hardware. The study found that LLM-based surrogates, improved through reinforcement learning, enable kernel searches to evaluate several times more candidates under the same GPU measurement budget, ultimately discovering faster kernels than equal-budget baselines. This approach addresses the growing bottleneck of on-device evaluation as LLM-driven kernel searches scale, suggesting a broader role for language models as virtual GPU models rather than solely as kernel generators.", "body_md": "# Computer Science > Machine Learning\n\n[Submitted on 29 May 2026]\n\n# Title:GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization\n\n[View PDF](/pdf/2605.31464)\n\nAbstract:GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth signal necessary for kernel search, they are costly, because each evaluation of a kernel requires compilation and repeated execution on a GPU. As improvements in LLM inference reduce the cost of writing novel kernels and LLM-driven searches scale to large search budgets, on-device evaluation becomes a bottleneck. To address this, we study how LLMs can serve as selective GPU surrogates for kernel evaluation, by forecasting the performance of proposed kernels. A useful surrogate should be accurate, and it should be selective, by knowing when it could be wrong, and deferring to the GPU. To evaluate surrogates, we measure whether their forecasts are accurate, calibrated, and practically useful for recovering fast kernels under limited GPU-measurement budgets. Next, we study whether reinforcement learning can improve forecast accuracy and confidence calibration. Our experiments demonstrate that LLMs can accurately forecast relative kernel performance, that their utility can be improved through reinforcement learning. Used inside a kernel search, the surrogate lets the search consider several times as many candidates under the same GPU evaluation budget, and that leads to finding faster kernels than an equal-budget baseline. These results suggest that LLMs can play a broader role in kernel optimization, by acting as virtual models of a GPU rather than solely as kernel generators for search.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/gpu-forecasters-language-models-as-selective-surrogates-for-kernel-optimization", "canonical_source": "https://arxiv.org/abs/2605.31464", "published_at": "2026-06-03 04:51:02+00:00", "updated_at": "2026-06-03 05:17:45.874016+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "artificial-intelligence", "neural-networks", "ai-research"], "entities": ["GPU", "LLM"], "alternates": {"html": "https://wpnews.pro/news/gpu-forecasters-language-models-as-selective-surrogates-for-kernel-optimization", "markdown": "https://wpnews.pro/news/gpu-forecasters-language-models-as-selective-surrogates-for-kernel-optimization.md", "text": "https://wpnews.pro/news/gpu-forecasters-language-models-as-selective-surrogates-for-kernel-optimization.txt", "jsonld": "https://wpnews.pro/news/gpu-forecasters-language-models-as-selective-surrogates-for-kernel-optimization.jsonld"}}