A PyTorch-style harness trainer. The LLM model stays frozen, the harness around it is being trained, including the prompts, context management, tools, and repair loop.
for loss in trainer.epochs(30):
loss.backward() # deposit the verdict on harness.grad
optimizer.step() # fast-forward HEAD to the winner, or reject
The harness is one editable file src/policy/core.py
. Each epoch, the Estimator in src/trainer/estimator.py
proposes one diff to it. The diff is incorporated into core.py
that will be measured against the current baseline on a panel of tasks, and a criterion decides whether the change (git commit) is promoted. git log
is the candidate promotion history. Every candidate, promoted or rejected, is kept under refs/candidates/
, and each measured run under refs/experiments/runs/
.
| PyTorch | Harness Training |
|---|---|
Parameter.data |
|
| HEAD's commit sha | |
| forward pass | run the task panel against the candidate harness |
loss.backward() |
|
write the candidate-vs-baseline verdict on to harness.grad |
|
optimizer.step() |
|
| fast-forward HEAD (promotion) or no-op (rejection) |
For more details, see the blog post: https://www.henrypan.com/blog/2026-07-18-harness-training
The "trained harness" is frozen, only the task-solving LLM for these evaluations is changed.
Train the harness on two Terminal-Bench tasks, then evaluate it on two held-out tasks.
Requirements: Python 3.13, Docker with linux/amd64
container support and the Compose plugin, the claude
CLI (it proposes the harness changes; swap in codex
with CodexAgentBackend
), local or remote OpenAI-compatible inference server with tool calling.
Example small models:
| Model | Quantized weights | Weight size | Example config |
|---|---|---|---|
Q4_K_M GGUFconfig/llm/qwen35_local.yaml
GPT-OSS-20BMXFP4 GGUFconfig/llm/local.yaml
Use either Ollama or llama.cpp or anything you like to expose the model through an OpenAI-compatible endpoint. Follow their platform-specific installation instructions.
The server must decode one request at a time (--parallel 1
for llama.cpp, OLLAMA_NUM_PARALLEL=1
for Ollama, whatever caps concurrent requests elsewhere): high concurrency batched decode is non-deterministic, so the run cannot be attributable to the candidate harness change. See Determinism. On the client side, we can still set max_rollout_concurrency
. You can enable batch decoding (higher concurrency) later, which I recommend following SGLang Deterministic Inference for larger scope training.
Point the quickstart at your server in config/llm/local.yaml, the one file both quickstart configs extend, then commit it — the trainer measures committed configs only.
model_name
is the id your server advertises, and tokenizer_name
is the HuggingFace Hub id, which is required for counting tokens for context length management.Did you read above? If so:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv sync
cp .env.example .env
uv run python examples/quickstart.py
Note
.env
is where keys live, and the variable must exist even when the server ignores auth — skipping thecp
above fails preflight withLOCAL_LLM_API_KEY is not set
. If your server checks keys, replace the shipped placeholder with the real one, or it answers401 Invalid API key
. - The default Terminal-Bench network cache reaches host services through
host.docker.internal
. See thenetwork-cache runbookfor the full host contract. You can decide whether to use the cache (default on to speed up quickstart). Training leaves the cache services and their volumes running. To stop them, keeping the cached data:docker compose -f src/env/netcache/docker-compose.caches.yml down
— add-v
to delete the volumes too. - Run it on a "experiment" branch: harness change promotions fast-forward your checkout,
git log
shows the candidate commit as your new baseline. - On first use, the model server downloads its weights and the quickstart downloads the task images. Runtime depends mainly on the selected model and hardware. Subsequent runs reuse the baseline (unless the harness drifted from baseline git commit SHA).
Run the quickstart in a disposable VM or container, a separate OS account, or a machine dedicated to agent workloads. The estimator runs as a host process with your inherited shell environment; see
criterion = StrictPareto()
optimizer = GreedyMonotonic()
trainer = Trainer(
config_path="config/train_harness.yaml",
estimator=AgenticEstimator(
backend=CodexAgentBackend(
trace_dir=Path("experiments/codex-traces"), model="gpt-5.6-sol"
)
),
criterion=criterion,
optimizer=optimizer,
)
for loss in trainer.epochs(30):
loss.backward() # Record the verdict.
optimizer.step() # Promote or reject.
The full walkthrough is in src/trainer/README.md.
In order for the training loop to produce useful signals across epochs. Each run's outcome must be attributable to the candidate only if everything else is deterministic: a seeded deterministic LLM inference engine, fixed container networks, deterministic environment, and a frozen network cache etc... This framework guarantees that in a couple of ways, more details in the blog post.
Always recommend to run on a isolated host machine to reduce risk. The uncertainty comes from the "Agent" proposing the harness change.
| path | what it is | start here to… |
|---|---|---|
src/policy/ |
src/trainer/
src/rollout/
src/env/
src/llm/
src/plugins/
config/
run_config.template.yaml
tests/
program.md
AgenticEstimator
hands its proposer and diagnoser each epoch- Training:
uv run python scripts/train.py
runs the full training loop withconfig/train_harness.yaml
. - Evaluation:
uv run python scripts/evaluate.py <config> <config2> ...
- Small end-to-end with training and evaluation:
examples/quickstart.py
Two environments ship out of the box, and one model provider — any OpenAI-compatible endpoint. A new benchmark is one DockerTaskEnv
subclass, a new provider is one CompletionBackend
subclass plus its registry entry. Use TerminalBenchEnv
and SweEnv
in src/env/, and
OpenAICompletionBackend
in , as the concrete references. Task images can take tens of GB of disk.
src/llm/
Bias the search toward one surface.
## Objective
+This run, propose only prompt-surface changes: system, initial, and repair prompts.
Hard-limit what candidates may edit. The patch surface is exactly the trained module's own file plus extra_patch_paths
— the framework rejects any diff outside it. To freeze part of the harness, move it out of the trained module; it is then unreachable to candidates.
training_target:
module: src.policy.core # must export build_policy / build_env_action
extra_patch_paths:
- tests/policy/test_core_impl.py # the only other file a candidate may write
Attack one failure mode. A panel of only the tasks that currently fail that way concentrates each epoch's signal on it:
environment:
$include: task_panels/tool_call_failures.yaml # example task panel
Re-fit the harness to a new model. The score is a property of model + harness, so a model swap means re-baseline and retrain.
$extends:
- llm/gptoss20_openrouter.yaml # was llm/qwen35_local.yaml
Promote cheaper solves. Tie-breaking is already on: every shipped entry point passes the benchmark's default secondary metrics. Pass your own tuple to change what breaks an exact solved-set tie, or define your own Criterion to replace the primary rule.
criterion = StrictPareto(secondary_metrics=(StepsUsedMetric(),))
criterion = NetTaskSolve() # placeholder criterion not in codebase
Drive proposals from any process. Nothing in the Estimator
contract says harness changes have to come from agents. You can define your own API, or an LLM panel with a judge, and each proposal goes through the same deterministic run.
class PatchQueue(Estimator):
"""A/B-test hand-written variants: you author the diffs, the loop measures them."""
def __init__(self, diffs: list[Path]):
self.diffs = iter(diffs)
def propose(
self, *, repo_root: Path, tracker: RunStore, target: TrainingTargetConfig
) -> None:
subprocess.run(["git", "apply", next(self.diffs)], cwd=repo_root, check=True)
def diagnose(self, result, *, repo_root, tracker, target) -> None:
pass # you are the diagnoser
MIT