# From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics > Source: > Published: 2026-09-13 12:54:24+00:00 From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics When a language model writes the next candidate, variation stops being a fixed kernel and becomes a model call conditioned on information you choose. This tutorial gives you the framework to classify that choice, build it, and decide whether it earns its cost. 1Apeiron Intelligence, Barcelona · 2Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra · 3Universitat Politècnica de Catalunya (UPC), Barcelona Large language models (LLMs) are increasingly being employed as variation operators in metaheuristics, generating or modifying candidate solutions, heuristics, or programs inside iterative search loops. This shift reframes variation as a model call conditioned on different types of information. We introduce an operator-level framework with two descriptors: (1) the type of prompt-conditioning information at variation time (Numeric, Symbolic, Linguistic), and (2) artifact persistence, identifying what survives the model call (Transient, Amortized, Transfer). The tutorial shows how to classify, build, and select these operators through a worked build template, a method survey, an evidence table, and a cost-aware decision guide. large language modelsmetaheuristicsvariation operatorsevolutionary computationcombinatorial optimizationprompt conditioningartifact persistence The question this tutorial answers One operator, three prompts. Which information should condition it? Consider a practitioner building an LLM-assisted heuristic for the Travelling Salesman Problem. The loop is the same in all three cases — ask the model for the next construction heuristic. What changes is what goes into the prompt. That single choice shifts cost across validation, evaluation and inference, and it is the axis the literature has not been organized around. Walkthrough The whole tutorial, read aloud A two-voice walkthrough of the paper, in English, twenty-three minutes long. It follows the same route as the text: the two descriptors, the placement rule and its audit, the build template with its validator and bounded repair, the two traced iterations, the map and its empty cells, the evidence table, and the decision guide. 01 — The organizing lens Conditioning channels A method is classified by the dominant channel in its variation prompt: the one that most drives the proposal. Most real methods are hybrids, so the full set of channels is reported and only the load-bearing one names the method. Numeric ``` h1: 427 h2: 411 h3: 419 ``` score-only trajectory over heuristics The encoded search information classical metaheuristics already consume: solution encodings, instance features, scalar scores, ranked parents, score trajectories. Evaluative content — how good is this? — easiest to check, least portable. Symbolic ``` python def step(T, U): c = nearest(T[-1], U) return T + [c] ``` executable construction rule Machine-interpretable artifacts with formal structure — code, syntax trees, formal rules, structured graphs — whose syntax or behaviour a parser, validator, compiler or executor can check. Denotational content — what does this compute or do? — verifiable by execution. Linguistic ``` "NN assigns tours to clusters; try a savings merge; then apply 2-opt repair" ``` strategy-level change to the heuristic Operative natural language: critiques, reflections, diagnoses, design principles, strategy notes that steer later proposals. Propositional content — why does this work, and what should change? — travels furthest, least mechanically checkable. The three channels sit side by side; none subsumes the others.Symbolic and Linguistic both add more explicit structure than Numeric, but they are incomparable with each other: code can be more structure-preserving than prose, while prose can express abstractions absent from the code. Fixed task headers and static problem statements do not by themselves make a method Linguistic. Artifact persistence The second descriptor asks a different question: what survives the model call? i Transient The LLM sits inside the loop. Every step pays a model call, and nothing outlives the run. ii Amortized The model is called offline to emit an artifact — a heuristic, a program — that then runs at near-zero runtime cost. iii Transfer Source-domain evidence is distilled into an inspectable artifact or principle, re-bound in a new domain without rediscovery. Still an emerging frontier — few methods reach this far. 02 — The map Where the field actually sits Representative methods placed on the two descriptors. Read each cell as a coordinate pair: what conditions the call, and what survives it. The empty cells are gaps in the current landscape, not impossibility claims. Numeric Symbolic Linguistic Transfer open open LAPTHiFo-Prompt Amortized open FunSearchLLaMEA EoHReEvo Transient LMXLLMOAEvoLLMOPROEvoPrompt MOLLEOLLEMAGI-LLM SGEMAEF † prompt-optimization boundary cases whose placement depends on prompt design. Methods cluster on and below the diagonal because higher persistence tends to require more domain-independent conditioning — a structural tendency the current body of methods reflects, not a law. 03 — Inside the tutorial Four things you can take to your own loop 01 A worked build template The full prompt → sample → parse-and-validate → bounded-repair → select loop, written out as an algorithm with an explicit repair cap, and mirrored by runnable code. 02 A method survey on one axis Representative and extended coverage of the literature classified by dominant conditioning channel and persistence — 30 methods placed against their primary sources. 03 An evidence table that refuses to be a leaderboard Each method's own headline result, tagged with whether it is peer-reviewed or preprint, and whether the comparison is matched, source-local, or qualitative. 04 A cost-aware decision guide A staged flowchart whose first branch is the null choice: add an LLM operator only when it adds a capability a classical operator does not. The test is a capability gap, not model availability. 04 — Evidence summary What each method actually reports Reported results are source-specific. Compare across rows only where the evidence column says matched — same evaluator, same harness. Everything else is a useful signal, not settled comparative evidence. Method Conditioning Emitted Call locus Evidence Reported result PR = peer-reviewed · pre = preprint · own = source-specific benchmark · matched = same evaluator · qual = qualitative only · † boundary case. BPP = bin packing problem · BBOB = the black-box optimization benchmark suite · p.p. = percentage points. 05 — Appendix results Placing every method: the drop-channel audit A classification is only as good as its rule. The audit applies one operation to every method: remove or neutralize a channel while keeping a valid scaffold, and see whether the operator still has anything to work with. This is what resolves the awkward cases — methods that emit code yet are steered by natural language. Method Channels present Dominance Drop-channel reasoning NL = natural language · AHD = automatic heuristic design, the offline design of a heuristic by search. Validation of the placement rule Two checks accompany the companion code — one for reproducibility, one for whether a channel is actually load-bearing. The second returned a negative result, and it is reported as such. 3independent LLM codersOne instance each from Claude Fable 5, GPT-5.5 and DeepSeek-V4-Pro, queried separately and shown only the placement rule and the channel descriptions. 9/12methods coded unanimouslyThe three divergences fall where the channel description is least determinate: OPRO and LMX, whose bare “solution” descriptions under-determine the representation, and ReEvo, where reflection and code-substrate readings compete. 0.73Gwet's AC1 agreementChance-corrected agreement across the three coders — the measure that discounts the agreement you would get by luck alone. The labelling is largely reproducible; its contested cases are exactly the annotated boundary ones. Load-bearing test — a negative result, reported A drop-channel ablation compared an EoH-style operator on online bin packing with and without its natural-language idea, holding parent code and scores fixed. On this easy task both conditions reached the optimum (mean gap 0%; 6 of 6 candidates optimal). The ablation did not demonstrate performance necessity for the linguistic channel on that task. EoH is still reported as Linguistic: the idea is the source-described design-intent signal that structurally distinguishes it from code-only heuristic design. Placement is reproducible; how much a present channel contributes is task-dependent. Extended coverage map A broader set of methods placed on the lens, beyond the representative ones, to show the field's breadth. Each entry is read against its primary source. It shows the shape of the field; the systematic census lives elsewhere. Entries marked adjacent fall outside the lens: they translate a specification rather than vary a candidate, and are listed only as foils. Method Dominant conditioning Persistence Conditions on → emits 06 — Companion code Runs offline. No API key, no network. A deterministic mock model returns completions from a fixed pool, so the whole build → sample → validate → repair → select loop executes end to end and reproduces exactly. An optional adapter is included for real model runs. Python 3.9+, standard library only. search.pythe reference build loop, with bounded repair llm.pypluggable interface: offline mock + optional real backend