Tutorial · Metaheuristics × Large Language Models
When a language model writes the next candidate, variation stops being a fixed kernel and becomes a model call conditioned on information you choose. This tutorial gives you the framework to classify that choice, build it, and decide whether it earns its cost.
1Apeiron Intelligence, Barcelona · 2Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra · 3Universitat Politècnica de Catalunya (UPC), Barcelona
Abstract
Large language models (LLMs) are increasingly being employed as variation operators in
metaheuristics, generating or modifying candidate solutions, heuristics, or programs inside
iterative search loops. This shift reframes variation as a model call conditioned on
different types of information. We introduce an operator-level framework
with two descriptors: (1) the type of prompt-conditioning information at variation time
(Numeric, Symbolic, Linguistic), and (2) artifact
persistence, identifying what survives the model call (Transient,
Amortized, Transfer). The tutorial shows how to classify, build, and
select these operators through a worked build template, a method survey, an evidence table,
and a cost-aware decision guide.
large language modelsmetaheuristicsvariation operatorsevolutionary computationcombinatorial optimizationprompt conditioningartifact persistence
The question this tutorial answers
Consider a practitioner building an LLM-assisted heuristic for the Travelling Salesman Problem. The loop is the same in all three cases — ask the model for the next construction heuristic. What changes is what goes into the prompt. That single choice shifts cost across validation, evaluation and inference, and it is the axis the literature has not been organized around.
Walkthrough
A two-voice walkthrough of the paper, in English, twenty-three minutes long. It follows the same route as the text: the two descriptors, the placement rule and its audit, the build template with its validator and bounded repair, the two traced iterations, the map and its empty cells, the evidence table, and the decision guide.
01 — The organizing lens
A method is classified by the dominant channel in its variation prompt: the one that most drives the proposal. Most real methods are hybrids, so the full set of channels is reported and only the load-bearing one names the method.
Numerich1: 427
h2: 411
h3: 419score-only trajectory over heuristics
The encoded search information classical metaheuristics already consume: solution encodings, instance features, scalar scores, ranked parents, score trajectories. Evaluative content — how good is this? — easiest to check, least portable.
Symbolicdef step(T, U):
c = nearest(T[-1], U)
return T + [c]executable construction rule
Machine-interpretable artifacts with formal structure — code, syntax trees, formal rules, structured graphs — whose syntax or behaviour a parser, validator, compiler or executor can check. Denotational content — what does this compute or do? — verifiable by execution.
Linguistic"NN assigns tours to clusters;
try a savings merge;
then apply 2-opt repair"strategy-level change to the heuristic
Operative natural language: critiques, reflections, diagnoses, design principles, strategy notes that steer later proposals. Propositional content — why does this work, and what should change? — travels furthest, least mechanically checkable.
The three channels sit side by side; none subsumes the others.
Symbolic and Linguistic both add more
explicit structure than Numeric, but they are incomparable with each other: code can
be more structure-preserving than prose, while prose can express abstractions absent from the
code. Fixed task headers and static problem statements do not by themselves make a
method Linguistic.
The second descriptor asks a different question: what survives the model call?
TransientThe LLM sits inside the loop. Every step pays a model call, and nothing outlives the run.
AmortizedThe model is called offline to emit an artifact — a heuristic, a program — that then runs at near-zero runtime cost.
TransferSource-domain evidence is distilled into an inspectable artifact or principle, re-bound in a new domain without rediscovery. Still an emerging frontier — few methods reach this far.
02 — The map
Representative methods placed on the two descriptors. Read each cell as a coordinate pair: what conditions the call, and what survives it. The empty cells are gaps in the current landscape, not impossibility claims.
NumericSymbolicLinguisticTransferAmortizedTransient† prompt-optimization boundary cases whose placement depends on prompt design. Methods cluster on and below the diagonal because higher persistence tends to require more domain-independent conditioning — a structural tendency the current body of methods reflects, not a law.
03 — Inside the tutorial
The full prompt → sample → parse-and-validate → bounded-repair → select loop, written out as an algorithm with an explicit repair cap, and mirrored by runnable code.
Representative and extended coverage of the literature classified by dominant conditioning channel and persistence — 30 methods placed against their primary sources.
Each method's own headline result, tagged with whether it is peer-reviewed or preprint, and whether the comparison is matched, source-local, or qualitative.
A staged flowchart whose first branch is the null choice: add an LLM operator only when it adds a capability a classical operator does not. The test is a capability gap, not model availability.
04 — Evidence summary
Reported results are source-specific. Compare across rows only where the evidence column says matched — same evaluator, same harness. Everything else is a useful signal, not settled comparative evidence.
| Method | Conditioning | Emitted | Call locus | Evidence | Reported result |
|---|
PR = peer-reviewed · pre = preprint · own = source-specific benchmark ·
matched = same evaluator · qual = qualitative only · † boundary case.
BPP = bin packing problem · BBOB = the black-box optimization benchmark suite ·
p.p. = percentage points.
05 — Appendix results
A classification is only as good as its rule. The audit applies one operation to every method: remove or neutralize a channel while keeping a valid scaffold, and see whether the operator still has anything to work with. This is what resolves the awkward cases — methods that emit code yet are steered by natural language.
| Method | Channels present | Dominance | Drop-channel reasoning |
|---|
NL = natural language · AHD = automatic heuristic design, the offline design of a heuristic by search.
Two checks accompany the companion code — one for reproducibility, one for whether a channel is actually load-bearing. The second returned a negative result, and it is reported as such.
Load-bearing test — a negative result, reported
A drop-channel ablation compared an EoH-style operator on online bin packing with and without its natural-language idea, holding parent code and scores fixed. On this easy task both conditions reached the optimum (mean gap 0%; 6 of 6 candidates optimal). The ablation did not demonstrate performance necessity for the linguistic channel on that task.
EoH is still reported as Linguistic: the idea is the source-described
design-intent signal that structurally distinguishes it from code-only heuristic design.
Placement is reproducible; how much a present channel contributes is task-dependent.
A broader set of methods placed on the lens, beyond the representative ones, to show the field's breadth. Each entry is read against its primary source. It shows the shape of the field; the systematic census lives elsewhere. Entries marked adjacent fall outside the lens: they translate a specification rather than vary a candidate, and are listed only as foils.
| Method | Dominant conditioning | Persistence | Conditions on → emits |
|---|
06 — Companion code
A deterministic mock model returns completions from a fixed pool, so the whole build → sample → validate → repair → select loop executes end to end and reproduces exactly. An optional adapter is included for real model runs. Python 3.9+, standard library only.
search.pythe reference build loop, with bounded repairllm.pypluggable interface: offline mock + optional real backendtsp_transient.pya transient, numeric-conditioned TSP operatorbpp_amortized.pyan amortized, code-emitting bin-packing operatorablation.pythe drop-channel ablation harness of the appendix# the three appendix experiments
python3 tsp_transient.py
python3 bpp_amortized.py
python3 ablation.py candidates.jsonEach script prints its own expected output, so a run either reproduces the numbers in the appendix or visibly does not.
07 — Cite
@article{chaconsartori2026semanticturn,
author = {Chac\'{o}n Sartori, Camilo and
Rodr\'{i}guez-Corominas, Guillem and
Blum, Christian},
title = {From Hand-Crafted to {LLM}-Based Variation Operators
in Metaheuristics: A Tutorial},
year = {2026},
note = {Preprint in preparation}
}Questions and corrections are welcome — camilo.chacon@apeironagents.tech