What breaks when you run coding agents on gpt-oss-120b locally

A developer built Stepswitch, an MIT-licensed orchestrator that splits coding tasks into steps and routes each to a suitable model, local first, with every change required to pass tests and a review by a model from a different family. On SWE-bench Verified Mini (50 django and sphinx issues, graded by the official harness), Stepswitch running gpt-oss-120b on a single workstation's Ollama resolved 24 of 49 compared instances (49.0%) versus 7 (14.3%) for mini-SWE-agent, but the developer attributes most of the gap to error handling: Ollama's HTTP 500 'error parsing tool call' response ended 24 of mini-SWE-agent's 49 runs, and on the 24 instances where neither run hit a server error the two agents resolved 7 and 6 respectively. The developer advises anyone running agents on gpt-oss through Ollama to catch that error and re-ask, or expect runs to end early.

I built Stepswitch https://github.com/sarthak-1111/stepswitch , a free, MIT-licensed orchestrator for coding agents. It splits a task into steps and sends each one to the model that suits it, local first. Every change has to pass your tests and a review by a model from a different family, and every number in the README is generated from evidence files that ssw prove re-creates. To see how it holds up on real issues, I ran it against mini-SWE-agent on SWE-bench Verified Mini: 50 django and sphinx issues, graded by the official harness on tests the agent never sees. Both agents used gpt-oss-120b on one workstation's Ollama, one run each, with the claims registered before the run. | | Stepswitch | mini-SWE-agent | |---|---|---| | Resolved 49 compared | 24 49.0% | 7 14.3% | | Runs ended by a server error | 1 | 24 | | Resolved where neither run hit a server error 24 instances | 7 | 6 | | Tokens per instance, counting cached | 1.61M | 2.37M | | Uncached tokens per instance | 142k | 46k | | Minutes per instance | 15.5 | 53.5 | One instance is left out for both because mini-SWE-agent's run hit the four-hour cap. The headline gap looks big, but the third row tells the real story: most of it is error handling, not smarts. When gpt-oss writes a tool call that Ollama can't parse, Ollama answers with HTTP 500 "error parsing tool call" . That one error ended 24 of mini-SWE-agent's 49 runs. Two things I learned about it: If you run any agent on gpt-oss through Ollama, catch this error and re-ask, or expect some runs to end early. num ctx on every Ollama request. On my own small suites 42 task runs , I also compared the local setup gpt-oss:120b for most steps, qwen2.5:32b as the reviewer with Claude Code: | | Passed | Tokens per task run | Uncached | Hosted tokens | Minutes | |---|---|---|---|---|---| | Stepswitch, local | 41/42 | 24.8k | 7.0k | 0 | 0.7 | | Claude Code via ssw proxy , local | 42/42 | 173k | 48.2k | 0 | 2.8 | | Claude Code, Sonnet 5.5 | 42/42 | 73.4k | 7.8k | all | 0.3 | | Claude Code, Opus 5.5 | 42/42 | 93.9k | 9.4k | all | 0.4 | These suites are small and easy, so treat them as a sanity check, not a benchmark. Predictions are committed in the harness's format, so you can grade them yourself, and ssw prove re-runs every claim's test. CI runs those checks on Linux, macOS and Windows. Repo: https://github.com/sarthak-1111/stepswitch https://github.com/sarthak-1111/stepswitch Feedback welcome, especially from anyone running agents on local models.