{"slug": "train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just", "title": "Train the Model Inside Your Own Agent Harness: What OpenHands and Snowflake Just Made Possible", "summary": "OpenHands and Snowflake AI Research published an approach that post-trains coding-agent models inside the team's existing production harness rather than re-implementing it in an RL framework. The OpenHands Agent SDK passes the agent's system prompt, tools, skills, turn budget and context length as configuration, a proxy records the model's own output tokens as training data, and Harbor task test scripts supply the reward, with rollouts and training running on Snowflake Cortex Sandboxes and Cortex Training (four H200s per run by default). The teams validated it with a custom code-localization harness on SWE-smith GitHub issues using a 10-turn budget and an F1-based reward.", "body_md": "Most teams customize their coding agent heavily: a tuned system prompt, a chosen set of tools, skills for recurring workflows, a turn budget that fits their repo. But the model underneath usually gets no such treatment. It was post-trained on someone else's generic agent setup, then dropped into yours — and the mismatch shows up as wrong tool calls, ignored skills, wasted steps, or agents that burn through their turn budget before finishing.\n\nA November 3 post from OpenHands and Snowflake AI Research proposes a different split of labor: instead of rebuilding your harness inside a reinforcement-learning framework, post-train the model inside the harness you already run.\n\nReinforcement learning for agents works by letting the model attempt tasks, scoring the outcome, and updating the weights toward what worked. The catch is that most RL frameworks bring their own agent loop, tool formats, and task structure. To train, you re-implement your production harness inside the training framework — extra engineering, plus a quiet risk that the thing you trained is not quite the thing you deploy.\n\nThe OpenHands approach removes that second implementation. With the OpenHands Agent SDK, your agent's configuration — system prompt, tools, skills, turn budget, context length — is passed as configuration, not recreated. Every training rollout is a real execution of your agent.\n\nThe mechanism is deliberately boring, which is the point. The agent does not know it is being trained: it sends ordinary chat-completion requests, and a proxy between the agent and the model records the exact tokens the model produces on each turn. Those tokens become the training data. Prompts and tool results are excluded from the loss, so the model learns from its own outputs, not from memorizing your instructions.\n\nTasks use the open Harbor format, which many teams already use for agent evaluation. A Harbor task is a directory with an `instruction.md`, an `environment/` sandbox definition, and a `tests/test.sh` script — and that test script's score becomes the reward signal. Each rollout runs in its own Snowflake Cortex Sandbox, with the agent's terminal and the grader inside the same sandbox, so the reward reflects what the agent actually did. Training compute comes from Snowflake Cortex Training.\n\nConfiguration maps directly onto familiar knobs: the system prompt via `OPENHANDS_SYSTEM_PROMPT_FILE`, tools via `OPENHANDS_TOOLS`, skills via `HARBOR_SKILL` paths, turn budget and context length via `MAX_TURNS` and `MAX_CONTEXT_LENGTH`, and the base model via `MODEL` — the example in the post uses Qwen/Qwen3.5-4B.\n\nRunning RL still involves operational work: allocating GPUs, setting batch sizes, launching jobs, watching for failed rollouts, releasing compute. OpenHands ships a `cortex-training-rl` skill (in `OpenHands/extensions`) that lets you describe the run in plain language and have the agent handle setup and execution:\n\n```\nUse /cortex-training-rl to post-train Qwen/Qwen3.5-4B on my Harbor tasks in ~/my-harbor-tasks/train and ~/my-harbor-tasks/eval with my agent's skill at ~/my-skill/SKILL.md, a 15-turn budget, for 50 steps.\n```\n\nThe agent configures the launcher, checks GPU availability on Cortex Training, starts the run in the background, reports reward metrics and failed rollouts at every step, plots the reward curve, and releases GPU resources when done. The skill also works from Claude Code and Cursor. Your local machine only runs the agent rollouts on CPU; model training and sampling run on Snowflake GPUs, four H200s per run by default.\n\nTo test the idea end to end, the teams built a custom code-localization harness: given a GitHub issue from a real Python repository in the SWE-smith dataset, the agent must identify the files, modules, and functions that need to change. The harness used a localization-specific system prompt, a terminal tool, a custom `localization_finish` tool for submitting answers, and a 10-turn budget. The reward was the sum of F1 scores at the file, module, and function levels, for a maximum of 3.0.\n\nThey trained Qwen3.5-4B in this harness for 25 steps at 64 rollouts per step. Mean reward climbed across training, and — the result that matters — the outcome was comparable to a purpose-built RL implementation. The full run, from Harbor tasks to a trained model, finished in a couple of hours.\n\nThat comparison is the honest benchmark for this approach. If training inside your existing harness only matched a bespoke setup, that is already a win, because the bespoke setup is the thing most teams will never build.\n\nThis is not a drop-in for every agent. A few details from the post are worth knowing before you try it:\n\n`file_editor` tool is not currently available in training runs.\nPost-training inside your harness makes sense when three things are true: you have a stable harness you plan to keep, you have graded tasks that represent real work, and your failures are behavioral — the model reaching for the wrong tool or ignoring a skill — rather than a lack of raw capability. In that situation, the model is not failing because it cannot code; it is failing because it has never practiced inside your setup. Training it there is the direct fix.\n\nIf those conditions do not hold yet, the cheaper sequence is still: fix the prompt, tighten the tools, build the task set, then train. What OpenHands and Snowflake changed is what happens after that sequence — post-training stops being a separate research project with its own framework and GPU cluster, and becomes one more step in the harness development loop.\n\nSources: OpenHands, \"Post-Train a Model for Your Own Agent Harness with OpenHands and Snowflake\" (November 3, 2026); Snowflake, Cortex Training announcement at Expedition 2026.", "url": "https://wpnews.pro/news/train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just", "canonical_source": "https://dev.to/lm_phlinh_b142146bb585/train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just-made-possible-flm", "published_at": "2026-11-04 08:18:16+00:00", "updated_at": "2026-11-04 08:23:00.604517+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "mlops", "developer-tools"], "entities": ["OpenHands", "Snowflake AI Research", "OpenHands Agent SDK", "Snowflake Cortex Training", "Snowflake Cortex Sandboxes", "Harbor", "Qwen3.5-4B", "SWE-smith"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just", "markdown": "https://wpnews.pro/news/train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just.md", "text": "https://wpnews.pro/news/train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just.txt", "jsonld": "https://wpnews.pro/news/train-the-model-inside-your-own-agent-harness-what-openhands-and-snowflake-just.jsonld"}}