{"slug": "harness-adapters-one-lora-lifts-qwen3-8-27b-from-55-5-to-63-5-across-three", "title": "Harness Adapters: One LoRA Lifts Qwen3.8-27B From 55.5% to 63.5% Across Three Coding Agents", "summary": "Hugging Face published a harness-adapter recipe on November 20, 2026, showing that a single merged LoRA trained with GRPO inside real coding-agent harnesses lifts Qwen3.8-27B's mean score across six harness and benchmark pairs from 55.5% to 63.5% with thinking on, and from 39.5% to 49.0% with thinking off while cutting tool calls by 53% to 73%. The super-adapter, released under Apache-2.0 in the harness-hub/qwen38-27b-harness-adapters collection, concatenates three rank-32 adapters into rank 96 and targets pi 1.5.2, OpenCode 2.2.7 and Hermes Agent v0.22.6, with training tasks drawn from SWE-smith, SWE-Gym via TaskTrove and Tasksmith. Of 442 scored tasks, the base model solved 151 every time and 120 never, leaving 171 solved only sometimes as the qualified GRPO training set.", "body_md": "Hugging Face published a recipe for tuning an open model to the coding agent that runs it, and a single merged adapter that works across three agents. The result: a Qwen3.8-27B super-adapter lifts the mean score across six harness and benchmark pairs from 55.5% to 63.5% with thinking on, per the [Hugging Face post on harness adapters](https://huggingface.co/blog/harness-adapters-at-scale?ref=explainx), published November 20, 2026.\n\nThe idea is simple. A harness is the wrapper that runs a model on tasks: its tools, prompts and loop. Models are not tuned for every harness. Train a small LoRA with reinforcement learning inside your harness, on your tasks, and you recover accuracy and efficiency. The authors write that combining RL on a custom task distribution with your favorite agent harness \"yields more accurate and efficient models.\"\n\nThis post explains the method, the numbers, how to run the adapter, and where the claims stop. It builds on our earlier coverage of the individual adapters: the [pi harness SWE-Bench Pro claim](https://explainx.ai/blog/qwen38-27b-swe-bench-pro-63-pi-harness-claimed-vs-verified-2026), the [OpenCode adapter](https://explainx.ai/blog/qwen38-27b-opencode-harness-adapter-lora-november-2026) and the [Hermes adapter](https://explainx.ai/blog/qwen38-27b-hermes-adapter-harness-hub-terminal-bench-58-8-lora-november-2026).\n\n## Harness adapters at a glance\n\n| Question | Answer | \n|---|---|\n| What is it? | A LoRA trained with RL (GRPO) inside a real agent harness | \n| Base model | Qwen/Qwen3.8-27B | \n| Harnesses | pi 1.5.2, OpenCode 2.2.7, Hermes Agent v0.22.6 | \n| Headline result | Thinking-on mean 55.5% to 63.5% across six pairs | \n| Thinking off | Mean 39.5% to 49.0%; 53 to 73% fewer tool calls | \n| License | Apache-2.0 (super-adapter model card) | \n| Rank | 96 (three rank-32 adapters concatenated) | \n| Where | Hugging Face collection harness-hub/qwen38-27b-harness-adapters | \n\n## What is a harness adapter?\n\nA harness adapter is a LoRA adapter, a small set of extra weights added on top of a frozen base model, trained with reinforcement learning while the model runs inside the real harness. Our [fine-tuning explainers](https://explainx.ai/blog/baseten-loops-serverless-sft-rl-training-glm-5-3-qwen3-6-per-token-billing-ga-what-docs-confirm-2026) cover the general idea. What is new here is the loop.\n\nHugging Face did not re-implement the harness for training. A proxy sits between the harness and vLLM and records the exact tokens the model produced. The trainer then learns from those tokens. That removes the usual mismatch between a training sandbox and the tool the developer actually runs.\n\nThe post puts the motivation plainly: \"Models are not fully optimized for all harnesses and the tasks you're doing.\"\n\n## How the training pipeline works\n\n### Tasks\n\nTask pipeline: Tasksmith, SWE-smith and SWE-Gym sources pass through an eligibility screen, Harbor packaging, openenv validate and a difficulty check with Qwen3.8-27B\n*Image: [Hugging Face](https://huggingface.co/blog/harness-adapters-at-scale?ref=explainx)*\n\nTasks come from three sources: SWE-smith (injected bugs in Python and Go), SWE-Gym via TaskTrove (real GitHub issues) and Tasksmith (generated from merged pull requests). Each task passes a metadata screen, `openenv validate` checks and a difficulty check against the base model.\n\nQwen3.8-27B results on 442 scored tasks, split by attempts solved and by source\n*Image: [Hugging Face](https://huggingface.co/blog/harness-adapters-at-scale?ref=explainx)*\n\nThe difficulty filter matters. Of 442 scored tasks, the base model solved 151 every time and 120 never. The 171 it solved only sometimes became the qualified set, because GRPO needs variation between attempts to learn anything. Our coverage of [Termgrade terminal RL environments](https://explainx.ai/blog/termgrade-1004-terminal-rl-environments-ai-and-weyaxi-release-2026) shows where part of the mix came from.\n\n### Training settings\n\n- LoRA on all linear layers.\n- GRPO via TRL's `AsyncGRPOTrainer` , with thinking off during rollouts.\n- Reward: the verifier's pass or fail, plus a 0.1 tool-efficiency bonus.\n- Learning rate 2e-5, chosen after a sweep over 1e-5, 5e-5 and 1e-4.\n- Scale: 598 cumulative steps, 78,114 samples and about 167M trained tokens, on a mixture of 750 harness-adapters tasks and 300 Termgrade tasks.\n\nThe tool-efficiency bonus explains a result that looks odd at first: the adapters are not only more accurate, they also use far fewer tokens. For the pi adapter, average tokens per task fell from 2.9K to 1.5K.\n\n### Safety plugin\n\nAgents running RL rollouts can do damage. Hugging Face used a safeagents plugin that blocked suspicious tool calls. Across the runs, 647 of about 1.3M tool calls were blocked. The plugin is published on the Hub.\n\n## What are the results?\n\nReinforcement learning as a ball following a track toward a cup, for Qwen3.8-27B harness adapter training\n\nSpecialist adapters, one per harness, thinking off, pass@1, as reported in the Hugging Face post:\n\n| Harness | Benchmark | Base | Specialist adapter | \n|---|---|---|---|\n| pi | SWE-bench Pro (hard 51) | 40.0% | 64.0% | \n| Hermes Agent | SWE-bench Pro (hard 51) | 29.4% | 43.1% | \n| OpenCode | SWE-bench Pro (hard 51) | 35.3% | 51.0% | \n| pi | Terminal-Bench 2.1 | 40.0% | 56.5% | \n| Hermes Agent | Terminal-Bench 2.1 | 44.7% | 58.8% | \n| OpenCode | Terminal-Bench 2.1 | 47.6% | 52.4% | \n\nThe text also quotes slightly different pi figures from another run (39.2% to 64.7%). Treat the table as one run, not a fixed constant.\n\n### The super-adapter\n\nThe three specialists were concatenated along the rank dimension: three rank-32 LoRAs make one rank-96 adapter. Each update was scaled by one half, which gave the best Terminal-Bench result. The adapters had cosine similarity of about 0.01 to 0.02, so they barely overlap, which is why a plain merge works. The model card says \"The merge itself involves no training.\"\n\nResults for the super-adapter, per the post and the [model card](https://huggingface.co/harness-hub/Qwen3.8-27B-super-adapter?ref=explainx):\n\n| Harness | Benchmark | Base, thinking on | Super-adapter, thinking on | \n|---|---|---|---|\n| pi | SWE-bench Pro (hard 51) | 74.0% | 74.0% | \n| Hermes Agent | SWE-bench Pro (hard 51) | 45.1% | 60.8% | \n| OpenCode | SWE-bench Pro (hard 51) | 47.1% | 66.7% | \n| pi | Terminal-Bench 2.1 | 52.9% | 57.6% | \n| Hermes Agent | Terminal-Bench 2.1 | 58.8% | 63.5% | \n| OpenCode | Terminal-Bench 2.1 | 54.9% | 58.5% | \n\nThinking on, the mean goes from 55.5% to 63.5%. It wins five of six pairs and ties one (pi on SWE-bench Pro). It uses 19 to 48% fewer tool calls per task. Thinking off, the mean goes from 39.5% to 49.0%; it newly solves 71 tasks and loses 34.\n\nNotice what the tie says. Where the base model with thinking was already strong (pi at 74.0%), the adapter adds nothing. The biggest gains land where the base is weakest: Hermes Agent and OpenCode on SWE-bench Pro.\n\n## Which harness gets the biggest gain?\n\nHermes Agent and OpenCode gain most on SWE-bench Pro with thinking on: Hermes from 45.1% to 60.8%, OpenCode from 47.1% to 66.7%. For a view of why harness choice moves scores this much, see our piece on [harness leaderboard reliability with Hermes Agent](https://explainx.ai/blog/joel-niklaus-harness-leaderboard-reliability-hermes-agent-2026). The Hugging Face post warns that results are only comparable inside the same harness, because agent settings differ across harnesses.\n\n## How to run the Qwen3.8-27B super-adapter\n\nThe model card gives a vLLM recipe. Serve the base model with LoRA enabled and a rank limit of 128, the next supported size above 96:\n\n```\nvllm serve Qwen/Qwen3.8-27B \\\n    --enable-lora --max-lora-rank 128 \\\n    --lora-modules qwen3.8-super=harness-hub/Qwen3.8-27B-super-adapter \\\n    --max-model-len 131072\n```\n\nThinking is on by default. To turn it off, pass `--default-chat-template-kwargs '{\"enable_thinking\": false}'`. Then point pi, Hermes Agent or OpenCode at `http://localhost:8000/v1`.\n\nYou can also load it with PEFT, merge it into a standalone checkpoint, or use the GGUF version for llama.cpp. Our [vLLM vs SGLang comparison](https://explainx.ai/blog/vllm-vs-sglang-inference-server-comparison-2026) helps you choose a server, and the [DSpine drafter post](https://explainx.ai/blog/qwen3-8-27b-dspine-b16-open-drafter-sglang-speculative-decoding-2026) covers another way to speed up the same base model.\n\n### Which adapter should I pick?\n\n| Your situation | Pick | \n|---|---|\n| One harness, thinking off | That harness specialist adapter | \n| Several harnesses, or thinking on | The super-adapter | \n| Laptop or llama.cpp | The OpenCode GGUF build | \n| Unsure about outside tasks | Test the base model next to the adapter first | \n\n## What are the limits?\n\nThe authors list several caveats, and they are worth reading before you adopt anything.\n\n- **Merge weight picked on the test set.** The super-adapter scale was chosen on Terminal-Bench 2.1, so those numbers are slightly optimistic.\n- **Out-of-distribution results are flat.** On AutomationBench (300 tasks), base scored 23.0%, pi 27.0%, OpenCode 25.3%, merged 23.7% and Hermes 17.7%. Differences are mostly within noise, except Hermes, which trails.\n- **Reasoning leakage.** With thinking off, the model often writes short reasoning-like text before tool calls.\n- **Harness differences.** pi and Hermes handle prior reasoning turns differently, which affects SWE-bench Pro results.\n- **Flawed verifiers.** One Terminal-Bench 2.1 verifier accepted known-wrong answers.\n- **Thinking-off specialists can lose to base with thinking on** in some cases.\n- **Small test slice.** The SWE-bench Pro set is a hard 51-task subset, so one task moves a score by about two points.\n\nNo dollar costs are reported. The post gives token and tool-call counts only. If you plan to use the weights commercially, read how [open-weight licenses work](https://explainx.ai/blog/open-weight-ai-model-licenses-explained-2026); the adapter is Apache-2.0 per its card, and the base model carries its own terms.\n\n## What this means for what you build or pay\n\n**Builders.** If you run an open 27B model behind a coding agent, a harness-specific LoRA is now a documented, reproducible step, not a research stunt. The data, tasks and trajectories are public, so you can reproduce the loop on your own harness and task mix. Check the result on your own repositories before trusting a 51-task benchmark. Our guide to [reading a Hugging Face model card](https://explainx.ai/blog/how-to-read-hugging-face-model-card-2026) helps with that.\n\n**Founders.** The efficiency numbers are the business story: 53 to 73% fewer tool calls with thinking off, and 19 to 48% fewer with thinking on. Fewer calls mean lower latency and lower serving cost per task. Because the gain is tied to a harness, it also raises switching costs if you later change agent tools.\n\n**Marketers and tool vendors.** Expect \"tuned for X harness\" to become a selling point. Ask for numbers on the same harness version, as the post lists pi 1.5.2, OpenCode 2.2.7 and Hermes Agent v0.22.6.\n\n## What is still open\n\nHugging Face has not shown that the gain transfers to tasks outside the training mix, and the AutomationBench result says it may not. It has not published a cost per task in dollars. And the merge trick is demonstrated on three adapters with near-orthogonal updates; more harnesses may interfere more. Independent reruns on other repositories would settle these.\n\n## Related reading\n\nOfficial sources: [Hugging Face blog](https://huggingface.co/blog/harness-adapters-at-scale?ref=explainx), [super-adapter model card](https://huggingface.co/harness-hub/Qwen3.8-27B-super-adapter?ref=explainx), [base model Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B?ref=explainx).\n\n*Figures reflect the Hugging Face post and model card as of November 20, 2026 and may change as the authors update them.*", "url": "https://wpnews.pro/news/harness-adapters-one-lora-lifts-qwen3-8-27b-from-55-5-to-63-5-across-three", "canonical_source": "https://explainx.ai/blog/hugging-face-harness-adapters-qwen38-27b-super-adapter-pi-hermes-opencode-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog", "published_at": "2026-11-20 00:00:00+00:00", "updated_at": "2026-11-20 15:16:04.533564+00:00", "lang": "en", "topics": ["ai-research", "ai-agents", "large-language-models", "machine-learning", "ai-tools"], "entities": ["Hugging Face", "Qwen3.8-27B", "pi 1.5.2", "OpenCode 2.2.7", "Hermes Agent v0.22.6", "SWE-smith", "SWE-Gym", "Tasksmith"], "also_reported_by": [], "takeaway": "Training a LoRA with GRPO inside the actual agent harness via a proxy between harness and vLLM raised Qwen3.8-27B's thinking-on mean from 55.5% to 63.5% across six harness-benchmark pairs and cut tool calls 53-73% with thinking off.", "recommendation": {"advice": "Filter RL tasks to those the base model solves only sometimes — 171 of 442 here — since GRPO needs variation between attempts, and train the LoRA inside the real harness via a proxy rather than a sandbox.", "playbook": "training-data", "playbook_url": "https://wpnews.pro/playbooks/training-data"}, "alternates": {"html": "https://wpnews.pro/news/harness-adapters-one-lora-lifts-qwen3-8-27b-from-55-5-to-63-5-across-three", "markdown": "https://wpnews.pro/news/harness-adapters-one-lora-lifts-qwen3-8-27b-from-55-5-to-63-5-across-three.md", "text": "https://wpnews.pro/news/harness-adapters-one-lora-lifts-qwen3-8-27b-from-55-5-to-63-5-across-three.txt", "jsonld": "https://wpnews.pro/news/harness-adapters-one-lora-lifts-qwen3-8-27b-from-55-5-to-63-5-across-three.jsonld"}}