When is a large language model a good calibration agent? Physics-labeled prompting for budgeted engineering simulator calibration

A hybrid workflow that uses a large language model as a warm-start for conventional optimizers reached target accuracy in 9 simulator runs, beating the LLM alone at 12 runs, Nelder-Mead at 17 runs and Bayesian optimization at more than 60 runs, according to arXiv paper 2610.22379v1. The study tested physics-labeled prompts on a cylinder-wake Large-Eddy Simulation and a damped-oscillator inverse problem, finding the first physics-informed LLM proposal improved the initial design in 23 of 25 trials and that median calibration error after five evaluations was 21.9 times lower than physics-free controls. The LLM's advantage was transient, disappearing by fifteen evaluations, and decoder variability produced a 9.8-fold performance spread, which the two-stage numerical continuation policy addresses by passing accumulated evaluations to a conventional optimizer.

arXiv:2610.22379v1 Announce Type: new Abstract: Calibrating expensive engineering simulators requires tuning model parameters to match reference data, a process heavily constrained by computational costs. This study investigates whether Large Language Models LLMs , conditioned on physical diagnostic descriptions of simulation errors, can propose informative parameter updates more efficiently than conventional derivative-free optimizers. We test this on a cylinder-wake Large-Eddy Simulation and a damped-oscillator inverse problem. Physics-labeled prompts are evaluated against four text ablations removed, anonymized, padded, misdirected and standard optimizers Nelder-Mead, Bayesian optimization, CMA-ES, and random search . Results demonstrate a strong early semantic advantage: the first physics-informed LLM proposal improves the initial design in 23 of 25 trials. After five evaluations, the median calibration error is 21.9 times lower than physics-free controls. However, this advantage is transient - disappearing by fifteen evaluations - and inherent LLM decoder variability yields a 9.8-fold performance spread. To address this, we introduce a hybrid workflow that uses the LLM as a warm-start, passing accumulated evaluations to a conventional optimizer. This two-stage numerical continuation policy reaches target accuracy in just 9 simulator runs, outperforming the LLM alone 12 runs , Nelder-Mead 17 runs , and Bayesian optimization 60 runs . Supported by sensitivity analysis validating the test cases, this hybrid physics-labeled LLM scheme proves highly effective as a calibration agent under strict evaluation budgets.