Bayesian Optimization with Rich Auxiliary Information via LLMs

Researchers developed three methods that use large language models to incorporate rich auxiliary information into Bayesian Optimization, according to arXiv paper 2609.19437v1. The methods consistently outperformed both standard Bayesian Optimization and existing LLM-based optimization approaches across hyperparameter optimization benchmarks and a real-world nuclear fusion optimization task. The work addresses optimization problems where training curves, expert notes, images, and prior knowledge about optima provide more information than function evaluations alone.

arXiv:2609.19437v1 Announce Type: new Abstract: Bayesian Optimization BO is widely used for optimizing expensive black-box functions, yet many real-world optimization problems contain substantially richer information than function evaluations alone. Examples include training curves in hyperparameter optimization, expert notes and images in scientific experimentation, and prior knowledge about where optima may lie. We show that large language models LLMs can effectively leverage such rich auxiliary information to guide optimization. Motivated by these findings, we develop three methods for incorporating auxiliary information into BO using LLMs. Across hyperparameter optimization benchmarks and a real-world nuclear fusion optimization task, our methods consistently outperform both standard BO and existing LLM-based optimization approaches. Our results demonstrate the effectiveness of LLMs for leveraging rich auxiliary information in BO.