{"slug": "cline-agents-harness-unlocks-benchmark-leading-performance-from-nvidia-nemotron", "title": "Cline Agents Harness Unlocks Benchmark-Leading Performance from NVIDIA Nemotron 3.5 Super VL", "summary": "Cline's agent harness scored 61.8% on Terminal-Bench 2.1 with NVIDIA's Nemotron-3.5 Super VL 120B A12B model running in thinking-off mode, the highest of four harnesses tested with that model, according to Cline. The three other open-source harnesses, labeled A, B, and C, scored 53.9%, 51.7%, and 44.9% respectively, with harnesses A and C using high reasoning or effort settings. Cline attributed the gap to harness features including a plan-before-implementation prompt, a stall detector that records structured events with error class, recoverability, and iteration number, and task-specific behavior checks, though Cline said it has not isolated which features account for the difference or measured token savings.", "body_md": "[Announcements](https://cline.ghost.io/tag/announcements/)\n\n# Cline Agents Harness Unlocks Benchmark-Leading Performance from NVIDIA Nemotron 3.5 Super VL\n\nExplore how NVIDIA’s latest Nemotron 3.5 Super model performs with Cline’s agent harness, delivering benchmark-leading results and powering real-world AI coding workflows.\n\nOpen weights models have become the new frontier of achieving intelligence more efficiently, and we are super excited to see how the latest NVIDIA  Nemotron 3.5 Super VL model continues to push the frontier of open weights models.\n\nWe’ve been testing NVIDIANemotron 3.5 Super VL in Cline, both on benchmarks and in our own day-to-day work. In our Terminal-Bench 2.1 evaluation, the model scored 61.8% with Cline and thinking mode turned off. That was the highest score among the four harnesses we tested with this model.\n\nWe’ve also built an internal agent around Nemotron that works with our team in Slack. It triages tasks, investigates user-reported bugs, and uses Cline to work through issues. The benchmark gives us a way to compare performance; the internal agent lets us see how the model handles the work our team needs done.\n\n## Unlocking powerful models with tailored harness\n\nA strong model matters. But the system wrapped around it can matter just as much.\n\nAn agent harness is the software around the model: the instructions it receives, the tools it can use, and the checks that guide its work. We’ve been refining those parts of Cline for open-weight models, including Nemotron 3.5 Super VL.\n\nIn this evaluation, Nemotron-3.5 Super VL 120B A12B scored 61.8% with Cline, compared with 53.9%, 51.7%, and 44.9% in three other open-source harnesses. We’ve labeled those harnesses A, B, and C in the chart below.\n\n*Terminal-Bench 2.1 scores for Nemotron-3.5 Super 120B. Cline: 61.8%; open-source harnesses A: 53.9%, B: 51.7%, C: 44.9%.*\n\nCline reached its score with thinking off; harnesses A and C used high reasoning or effort settings. This comparison shows how much results can vary with the software around the same model. It doesn’t isolate which Cline features account for the difference or measure how much compute they save.\n\n## What we changed in Cline\n\n### Plan before changing code\n\nCline prompts the model to plan before implementation: understand the task, choose an approach, then start making changes. That gives the model an explicit sequence to follow as it works.\n\nWe think this structure can help the model spend less effort deciding what to do next. The result with thinking off is encouraging, but we haven’t measured token savings or isolated the effect of the planning instruction. That would take token-usage data and a comparison with the instruction enabled and disabled.\n\n### Detect when an approach has stalled\n\nAn agent can keep repeating an action long after it stops being useful. Cline’s stall detector can end that attempt and record a structured event, including the error class, whether it is recoverable, and the iteration number.\n\nThose details make failed runs easier to investigate. In the other runs we reviewed, we often had to read through a sequence of tool calls and identify the repetition ourselves. A recorded stall gives us a specific place to start debugging.\n\n### Check the behavior the task depends on\n\nTwo very specific tasks from the evaluation show what this looks like in practice.\n\nOn chess-best-move, Cline parsed the board into FEN notation and ran an exhaustive checkmate search. It found two mates in one, e2e4 (Qe4#) and g2g4 (g4#), in a run spanning 58 log lines. Harness A made 29 tool calls and reopened the board image at least six times before timing out.\n\nOn headless-terminal, harnesses A and B both ran real tests. Cline completed its checks in five commands, using an itemized checklist that covered both whether Ctrl-C delivered a signal and whether the shell survived afterward. The narrower checks in the other runs missed the behavior that eventually broke.\n\nThese runs help us understand the score. They also give us concrete things to improve: how the agent chooses an approach, recognizes a dead end, and verifies that its fix works.\n\n## Using Nemotron inside Cline\n\nOur internal Nemotron-based agent lives in Slack and runs Cline under the hood. It can spin up sandboxes and containers, and call more advanced models when a task needs them. We use it to triage internal tasks and investigate and fix issues reported by users.\n\nOne example is a Windows diagnostics export failure. A teammate asked the agent to investigate a report in which Cline tried to create an existing Downloads folder, hit an EEXIST error, and failed to write the diagnostic report.\n\n*A teammate asks our internal agent to investigate a Windows diagnostics export failure.*\n\nThat is the kind of work we are super excited to see Nemotron super being able to help with: taking a user report, tracing the failure, and giving the team a useful next step. We’re encouraged by what Nemotron 3.5 Super VL can already do, and we’re continuing to refine Cline through benchmark runs and the issues our team handles every day!", "url": "https://wpnews.pro/news/cline-agents-harness-unlocks-benchmark-leading-performance-from-nvidia-nemotron", "canonical_source": "https://cline.ghost.io/nvidia-nemotron-3-5-super-vl-cline-agent-harness/", "published_at": "2026-10-15 15:25:52+00:00", "updated_at": "2026-10-15 15:53:58.419784+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "developer-tools"], "entities": ["Cline", "NVIDIA", "Nemotron 3.5 Super VL", "Nemotron-3.5 Super VL 120B A12B", "Terminal-Bench 2.1", "Slack"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/cline-agents-harness-unlocks-benchmark-leading-performance-from-nvidia-nemotron", "markdown": "https://wpnews.pro/news/cline-agents-harness-unlocks-benchmark-leading-performance-from-nvidia-nemotron.md", "text": "https://wpnews.pro/news/cline-agents-harness-unlocks-benchmark-leading-performance-from-nvidia-nemotron.txt", "jsonld": "https://wpnews.pro/news/cline-agents-harness-unlocks-benchmark-leading-performance-from-nvidia-nemotron.jsonld"}}