{"slug": "what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally", "title": "What breaks when you run coding agents on gpt-oss-120b locally", "summary": "A developer built Stepswitch, an MIT-licensed orchestrator that splits coding tasks into steps and routes each to a suitable model, local first, with every change required to pass tests and a review by a model from a different family. On SWE-bench Verified Mini (50 django and sphinx issues, graded by the official harness), Stepswitch running gpt-oss-120b on a single workstation's Ollama resolved 24 of 49 compared instances (49.0%) versus 7 (14.3%) for mini-SWE-agent, but the developer attributes most of the gap to error handling: Ollama's HTTP 500 'error parsing tool call' response ended 24 of mini-SWE-agent's 49 runs, and on the 24 instances where neither run hit a server error the two agents resolved 7 and 6 respectively. The developer advises anyone running agents on gpt-oss through Ollama to catch that error and re-ask, or expect runs to end early.", "body_md": "I built [Stepswitch](https://github.com/sarthak-1111/stepswitch), a free, MIT-licensed orchestrator for coding agents. It splits a task into steps and sends each one to the model that suits it, local first. Every change has to pass your tests and a review by a model from a different family, and every number in the README is generated from evidence files that `ssw prove` re-creates.\n\nTo see how it holds up on real issues, I ran it against mini-SWE-agent on SWE-bench Verified Mini: 50 django and sphinx issues, graded by the official harness on tests the agent never sees. Both agents used gpt-oss-120b on one workstation's Ollama, one run each, with the claims registered before the run.\n\n|  | Stepswitch | mini-SWE-agent | \n|---|---|---|\n| Resolved (49 compared) | 24 (49.0%) | 7 (14.3%) | \n| Runs ended by a server error | 1 | 24 | \n| Resolved where neither run hit a server error (24 instances) | 7 | 6 | \n| Tokens per instance, counting cached | 1.61M | 2.37M | \n| Uncached tokens per instance | 142k | 46k | \n| Minutes per instance | 15.5 | 53.5 | \n\nOne instance is left out for both because mini-SWE-agent's run hit the four-hour cap.\n\nThe headline gap looks big, but the third row tells the real story: most of it is error handling, not smarts.\n\nWhen gpt-oss writes a tool call that Ollama can't parse, Ollama answers with HTTP 500 (\"error parsing tool call\"). That one error ended 24 of mini-SWE-agent's 49 runs.\n\nTwo things I learned about it:\n\nIf you run any agent on gpt-oss through Ollama, catch this error and re-ask, or expect some runs to end early.\n\n`num_ctx` on every Ollama request.\nOn my own small suites (42 task runs), I also compared the local setup (gpt-oss:120b for most steps, qwen2.5:32b as the reviewer) with Claude Code:\n\n|  | Passed | Tokens per task run | Uncached | Hosted tokens | Minutes | \n|---|---|---|---|---|---|\n| Stepswitch, local | 41/42 | 24.8k | 7.0k | 0 | 0.7 | \n| Claude Code via `ssw proxy` , local | 42/42 | 173k | 48.2k | 0 | 2.8 | \n| Claude Code, Sonnet 5.5 | 42/42 | 73.4k | 7.8k | all | 0.3 | \n| Claude Code, Opus 5.5 | 42/42 | 93.9k | 9.4k | all | 0.4 | \n\nThese suites are small and easy, so treat them as a sanity check, not a benchmark.\n\nPredictions are committed in the harness's format, so you can grade them yourself, and `ssw prove` re-runs every claim's test. CI runs those checks on Linux, macOS and Windows.\n\nRepo: [https://github.com/sarthak-1111/stepswitch](https://github.com/sarthak-1111/stepswitch)\n\nFeedback welcome, especially from anyone running agents on local models.", "url": "https://wpnews.pro/news/what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally", "canonical_source": "https://dev.to/sarthak_verma_2e1ad04aa0f/what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally-35g", "published_at": "2026-10-27 13:38:01+00:00", "updated_at": "2026-10-27 13:42:30.689186+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "developer-tools", "ai-infrastructure"], "entities": ["Stepswitch", "mini-SWE-agent", "gpt-oss-120b", "Ollama", "SWE-bench Verified Mini", "Claude Code", "qwen2.5:32b"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally", "markdown": "https://wpnews.pro/news/what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally.md", "text": "https://wpnews.pro/news/what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally.txt", "jsonld": "https://wpnews.pro/news/what-breaks-when-you-run-coding-agents-on-gpt-oss-120b-locally.jsonld"}}