Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost Hugging Face researchers trained one LoRA adapter per agent harness for Qwen3.8-27B using reinforcement learning inside the harness itself, raising the model's Terminal-Bench 2.1 pass@1 score under the pi harness from 39% to 55% while cutting average token usage from 2.9K to 1.5K per task. The adapters were trained for three harnesses — Pi 1.5.2, OpenCode 2.2.7 and Hermes Agent v0.22.6 — with datasets, training and agent rollouts run on the Hub in Jobs using TRL's AsyncGRPOTrainer, OpenEnv and harbor. The three LoRAs were also merged into a single super adapter that runs unchanged under all three harnesses, and all artifacts are published in a Hugging Face collection. Image-Text-to-Text • 28B • Updated • 6.45M • 20.7k https://huggingface.co/Qwen/Qwen3.8-27B Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost Update on GitHub https://github.com/huggingface/blog/blob/main/harness-adapters-at-scale.md Combining RL on a custom task distribution with your favorite agent harness yields more accurate and efficient models for that target harness and task distribution. Models are not fully optimized for all harnesses and the tasks you’re doing. Instead, they target general performance. So when you combine a model and harness, you’re leaving performance and efficiency on the table for the stuff you’re actually doing. With RL, you can train a custom harness adapter that squeezes everything out of your model for your harness and task combination. Take Qwen3.8-27B, its Terminal-Bench 2.1 score with the pi harness is 39% which uses on average 2.9K tokens per task. If we freeze those weights and add a custom RL-trained LoRA adapter we can get 55% pass@1, with average token usage of 1.5K. Most of this is down to the model reasoning less, calling less tools, and performing better first time. Our earlier work multi-harness RL https://huggingface.co/spaces/FineEnvs/multi-harness-rl , harness adapters https://huggingface.co/blog/harness-hub/harness-adapters showed that RL on your own task distribution, inside your own harness, gives a more accurate and more efficient model. Those experiments covered data-analysis and code-building tasks. Here we wanted to push this further by exploring whether the result holds at 27B, on harder real-world software engineering tasks, across three harnesses: Pi, OpenCode and Hermes Agent. And most of all, share a fully reusable recipe, so you can reproduce this on your own task, harness, model combinations. TL;DR - We trained one LoRA adapter for Qwen3.8-27B per agent harness pi, Hermes Agent, OpenCode , with RL inside the harness itself. - Datasets, training and agent rollouts all happen on the Hub in Jobs using TRL's AsyncGRPOTrainer , OpenEnv and harbor. - Each harness adapter beats the base model under its own harness on both benchmarks, for both performance and efficiency. - The three LoRAs are also merged into a single super adapter that runs unchanged under pi, Hermes Agent and OpenCode. - All artifacts adapters, GGUFs, RL tasks, safeagents plugin can be browsed through this Hugging Face collection https://huggingface.co/collections/harness-hub/qwen38-27b-harness-adapters . Table of Contents Training inside agent harnesses We leverage the Harbor integration in OpenEnv https://huggingface.co/docs/openenv/environments/harbor and TRL’s OpenEnv https://huggingface.co/docs/trl/openenv guide to train the model inside the harnesses we chose for the final training. We have built ready-to-use docker images for all our RL tasks that come with the target harnesses already pre-installed inside. In our final docker images, we use Pi 1.5.2 , OpenCode 2.2.7 and Hermes Agent v0.22.6 . The diagram below from the aforementioned blogpost explains well the entire protocol: Most RL frameworks train a re-implementation of the agent loop: the agent is written against the framework's own interface, such as verl's AgentLoop https://verl.readthedocs.io/en/latest/advance/agent loop.html , SkyRL-Gym https://github.com/NovaSky-AI/SkyRL environments, rLLM https://github.com/rllm-org/rllm 's agent and environment classes, slime https://github.com/THUDM/slime 's custom rollout functions, or ART https://github.com/OpenPipe/ART 's client inside your own Python agent. Here the harness runs exactly as it ships, and the trainer learns from the exact tokens it produced, recorded by a proxy between the harness and vLLM. Recent work moves in the same direction: Agent Lightning v1.0 https://huggingface.co/papers/2608.17528 keeps the deploy-time harness in the loop through a proxy "harnessed agentic RL" , and SkyRL's Harbor integration https://novasky-ai.notion.site/skyrl-harbor trains on Harbor trials, converting their chat histories into tokens for the trainer. Generating custom RL tasks on Hugging Face For this recipe we created a custom dataset for agentic coding with the Hub, OpenEnv, and Repo2RLEnv https://github.com/huggingface/Repo2RLEnv . We combined together task retrieval of existing RL envs, task generation with Repo2RLEnv, task validation with OpenEnv, and sampling into a single data pipeline. This gave us the highest quality set of validated and relevant tasks to improve Qwen3.8-27B on agentic coding. In this section, we’ll walk through how we built it, so you can try it out on your own domains. RL training needs tasks the agent can attempt, a verifier that scores each attempt, and evidence that the verifier is right. A verifier that accepts wrong answers teaches the wrong behavior. A task that is already feasible will not teach the model anything. This section describes how we built the task set from three sources: two on the Hugging Face Hub and a third generated with Tasksmith, a pipeline in Repo2RLEnv. Every task passes an eligibility screen with openenv validate , and the same difficulty check against the base model, Qwen3.8-27B. Three sources Each task uses the Harbor format, which Terminal-Bench 2.1 also uses. A broad search of the Hub for Harbor environments returned about 59,000 tasks, but they included duplicates, evaluation sets and tasks marked as not for training. We filtered down to just two Hub sources built for training and generated a third set ourselves: - SWE-smith Python https://huggingface.co/datasets/SWE-bench/SWE-smith-py , Go https://huggingface.co/datasets/SWE-bench/SWE-smith-go injects bugs into real Python and Go repositories, with an LLM rewrite or a mutation such as a flipped operator. The tests that catch each bug become the task's tests. It adds volume. - SWE-Gym, packaged as Harbor tasks in TaskTrove https://huggingface.co/datasets/open-thoughts/TaskTrove , turns real GitHub issues and their merged fixes into tasks. It adds real-world bugs. - Tasksmith https://huggingface.github.io/Repo2RLEnv/pipelines/tasksmith/ generates tasks from merged pull requests. An LLM agent writes the instruction and tests, and Tasksmith checks the result: the reference and a valid alternative must pass, while the unfixed code and a known-wrong solution must fail. We used Tasksmith to target gaps. We mapped the skills that our target benchmarks test Terminal-Bench 2.1, the hard subset of SWE-bench Pro, and DeepSWE , then chose pull requests that exercise them, such as HTTP streaming, filesystem operations and serialization. From 30 merged pull requests, Tasksmith produced 24 tasks in 10 Python repositories that passed its checks, 17 of them on a second run. Its checks do not replace ours, so these tasks go through the same pipeline as the rest. Validation with openenv validate A metadata screen runs first. It keeps tasks from training splits with permissive licenses. It drops duplicates, tasks marked as not for training, the 253 tasks in our target benchmarks, and any task from a repository those benchmarks use. We then ran the semantic checks from openenv validate on every task in fresh trials: - The reference solution earns the maximum reward semantic.oracle max . - Doing nothing scores clearly lower semantic.floor gap . - No canary scores more than doing nothing semantic.canary floor . - A replay of the reference gives the same reward runtime.episode determinism . Each trial ran as its own Hugging Face Job, which let us validate dozens of tasks at once. openenv validate offers the same option for any Harbor task: openenv validate path/to/task --sandbox hf-sandbox 858 of 1,127 Hub candidates passed. Wrong-answer controls caught defects, including a Tasksmith task that accepted an incomplete implementation and a Terminal-Bench 2.1 verifier regex-log that accepted known-wrong answers. Most failures were infrastructure problems, such as images that did not start. Failed tasks stay in a rejected split with the failing check recorded. Difficulty check with the base model A valid task can still teach nothing. GRPO group relative policy optimization scores each attempt against the other attempts at the same task, so a task where every attempt passes, or every attempt fails, gives no learning signal. Before training, Qwen3.8-27B attempted each task 4 times, or 8 times for a stratified sample of six tasks per repository. Of 442 scored tasks, Qwen3.8-27B solved 151 every time and 120 never. The 171 tasks it solved only sometimes form the qualified set. The figure below shows how the scores pile up at both ends and how they differ by source. The model solves most Go tasks every time, fails every attempt on more than a third of the SWE-smith Python tasks, and lands in the middle on 73% of the SWE-Gym tasks. Qwen3.8-27B results on 442 scored tasks. Green marks the qualified tasks, which the model solves on some attempts but not all. This example shows how we filtered and validated 1,127 tasks to 278 which we included directly in the final dataset. The pipeline then took the remaining 522 rejected tasks, repaired and complexified them with Tasksmith and repeated validation. Testing the filters with short training runs The pipeline applies two filters: validation removes broken tasks, and the difficulty check keeps tasks the base model solves only sometimes. This section tests whether those filters change what the model learns. We trained four short runs of up to 50 steps, each on tasks from a different stage of the pipeline, and compared them with the base model on Terminal-Bench 2.1. Each run trains Qwen3.8-27B with LoRA and GRPO, using TRL's AsyncGRPOTrainer , on 32 tasks. All four use the same agent harness OpenCode with up to 48 model calls , the same settings and one 8×H200 Hugging Face Job, and the reward is the verifier's pass or fail. The task sets differ in how they were filtered: - Unvalidated : 24 accepted tasks plus 8 that failed validation. - Validated : 32 accepted tasks, chosen without regard to difficulty. - Unqualified : 32 accepted tasks that the base model always or never solves. - Qualified : 32 accepted tasks that the base model solves on some attempts. The figure below shows how the tasks impacted model learning. The table lists each run's steps and the checkpoint we evaluate. The chart tracks training reward across subsets of tasks with differing validation status. The dataset harness-hub/agentic-coding-envs holds 871 tasks from 35 repositories. Hub tasks are capped at 40 per repository, so no single project dominates. The table below shows what each source contributes. The dataset has two views: every accepted task, and the qualified subset for GRPO training. Rejected attempts stay in a separate config. | Source | Task type | Tasks | Qualified | |---|---|---|---| | SWE-smith | Injected bugs in Python and Go | 717 | 144 | | SWE-Gym TaskTrove | Real GitHub issues | 130 | 27 | | Tasksmith | Generated from merged pull requests | 24 | 22 | | Total | | 871 | 171 | AsyncGRPOTrainer for more efficient GRPO training We leveraged the stable version of AsyncGRPOTrainer from TRL, with the main advantage of the trainer being that it does not wait until all rollouts are completed before moving to the next training optimization step. For more detailed reading on the trainer and its building blocks, please refer to the AsyncGRPO blogpost https://huggingface.co/spaces/trl-lib/async-grpo-stable . Training setup To obtain our final harness adapters, we trained LoRA adapters on top of Qwen/Qwen3.8-27B https://huggingface.co/Qwen/Qwen3.8-27B using TRL's AsyncGRPOTrainer . The adapters were attached to the model using the all-linear flag. To maximize the training efficiency, thinking mode was turned off during rollout stages, and we evaluated our final adapter with and without thinking for broader comparison. The final reward function consists of the verifier's correctness, combined with a small tool-efficiency bonus 0.1, derived from early experiments from the previous iteration of harness adapters https://huggingface.co/blog/harness-hub/harness-adapters which showed that 0.1 worked best . We performed a small LR sweep on 50 steps using Termgrade environments 1e-5, 5e-5 and 1e-4 and noticed that 1e-5 barely moved training correctness from 0.52 to 0.50 , 1e-4 collapsed the rollouts instead of improving them turns per rollout moved from 18 to 6, with entropy rising , and only 5e-5 clearly improved from 0.49 to 0.60 by step 40 . Longer 5e-5 runs then showed signs of instability collapsing rollout lengths, rising entropy , so we settled on 2e-5 for the final runs. Training infrastructure Each harness adapter was trained using NVIDIA 2xB200 GPUs, with one GPU allocated for the vLLM server and the other allocated for the training process. We first performed some stress testing to maximize the amount of RL tasks that can be trained within a single optimization step. For that, we ‘maxed out’ the --token-budget argument from AsyncGRPOTrainer to its limit for the setup that we had. The chart below shows the evolution of the number of groups per step for each harness in blue: Pi, green: OpenCode and orange: Hermes . As the model gets better and better during training, it is able to fit more and more groups during each training step, which diversifies the signals that the model gets at each optimization step. Final mixture The final data mixture consists of 750 tasks from harness-adapters RL environments https://huggingface.co/datasets/harness-hub/harness-adapters and 300 tasks from termgrade https://huggingface.co/datasets/ai-and/termgrade-environments . In order to maximize the diversity of the sampled tasks during training, we have slightly customized the data sampling strategy, which will be explained below: Before training, the tasks are put in a fixed order that interleaves them by family i.e., their repository for our generated RL environments. For Termgrade, we just sorted them by name since each task is considered as its own family , so every training step mixes as many repositories as possible while following the distribution of the mixture. safeagents in the loop We also plug the safeagents https://github.com/huggingface/safeagents tool, in order to block any misbehavior from the model during training. The final plugin which consists of the regex and pattern rules specific to the RL tasks and test path is hosted in harness-hub/qwen-safeagents-plugin https://huggingface.co/harness-hub/qwen-safeagents-plugin and can be used out of the box using the latest version of safeagents in case anyone wants to reproduce the training setup. The number of blocked tools from safeagents decreases consistently during training, which indicates that the model gets ‘safer and safer’ after each optimization step. Below is a diagram of the safeagents blocked calls per 1000 tool calls, by training step, for each harness adapter. Super-adapter: merging all adapters into one Merging models often combines their capabilities while keeping most of what each one could do. Once the three harness adapters were trained, we asked two questions. Does merging them cost accuracy? And can a single "super-adapter" be accurate and efficient under every harness? We started by comparing the weight updates of the three adapters, layer by layer. The cosine similarity between any two of them is about 0.01–0.02: each adapter pushed the model in a nearly orthogonal direction during training. That's the best case for merging, since the updates barely interfere. We concatenate the three LoRA adapters along the rank dimension PEFT's combination type="cat" . The merged adapter has rank 96 3 × 32 and applies exactly w · ΔW pi + ΔW hermes + ΔW opencode . Prior work on merging task arithmetic https://huggingface.co/papers/2212.04089 , TIES https://huggingface.co/papers/2306.01708 , KnOTS https://huggingface.co/papers/2410.19735 finds that scaling each update below 1 works best, so we compared three weights on all 89 Terminal-Bench 2.1 tasks, under each harness, with the step-120 adapters: - ⅓ the average : each harness keeps only a third of its own update. - ½: each update at half strength. - 1 the plain sum : every update applied in full. | Weight | pi | hermes | opencode | All three harnesses | Tool calls per task | |---|---|---|---|---|---| | base model | 39.1% | 46.0% | 44.8% | 43.3% | 49.2 | | ⅓ average | 49.4% | 51.2% | 47.1% | 49.2% | 35.6 | | ½ | 48.8% | 53.5% | 55.2% | 52.5% | 25.9 | | 1 sum | 40.2% | 62.8% | 47.1% | 50.0% | 13.4 | The plain sum over-applies the updates. The agents become very short 13 tool calls per task, against 49 for the base model , and while Hermes peaks at 62.8%, pi falls back to the base model's level. The average dilutes each harness's own update. ½ gives the best balance, so we built the final super-adapter from the released checkpoints pi step 160, hermes step 160, opencode step 150 at weight ½. Adapter merging evaluation results Under each harness, on the tasks every run graded 50–51 on SWE-bench Pro, 82–85 on Terminal-Bench 2.1 , one attempt per task in parentheses, the efficiency gain with respect to the base model : | Harness | Benchmark | Qwen3.8-27B https://huggingface.co/Qwen/Qwen3.8-27B | Qwen3.8-27B-