I built Stepswitch, a free, MIT-licensed orchestrator for coding agents. It splits a task into steps and sends each one to the model that suits it, local first. Every change has to pass your tests and a review by a model from a different family, and every number in the README is generated from evidence files that ssw prove re-creates.
To see how it holds up on real issues, I ran it against mini-SWE-agent on SWE-bench Verified Mini: 50 django and sphinx issues, graded by the official harness on tests the agent never sees. Both agents used gpt-oss-120b on one workstation's Ollama, one run each, with the claims registered before the run.
| | Stepswitch | mini-SWE-agent |
|---|---|---|
| Resolved (49 compared) | 24 (49.0%) | 7 (14.3%) |
| Runs ended by a server error | 1 | 24 | | Resolved where neither run hit a server error (24 instances) | 7 | 6 | | Tokens per instance, counting cached | 1.61M | 2.37M | | Uncached tokens per instance | 142k | 46k | | Minutes per instance | 15.5 | 53.5 |
One instance is left out for both because mini-SWE-agent's run hit the four-hour cap.
The headline gap looks big, but the third row tells the real story: most of it is error handling, not smarts.
When gpt-oss writes a tool call that Ollama can't parse, Ollama answers with HTTP 500 ("error parsing tool call"). That one error ended 24 of mini-SWE-agent's 49 runs.
Two things I learned about it:
If you run any agent on gpt-oss through Ollama, catch this error and re-ask, or expect some runs to end early.
num_ctx on every Ollama request.
On my own small suites (42 task runs), I also compared the local setup (gpt-oss:120b for most steps, qwen2.5:32b as the reviewer) with Claude Code:
| Passed | Tokens per task run | Uncached | Hosted tokens | Minutes | |
|---|---|---|---|---|---|
| Stepswitch, local | 41/42 | 24.8k | 7.0k | 0 | 0.7 |
Claude Code via ssw proxy , local |
42/42 | 173k | 48.2k | 0 | 2.8 |
| Claude Code, Sonnet 5.5 | 42/42 | 73.4k | 7.8k | all | 0.3 |
| Claude Code, Opus 5.5 | 42/42 | 93.9k | 9.4k | all | 0.4 |
These suites are small and easy, so treat them as a sanity check, not a benchmark.
Predictions are committed in the harness's format, so you can grade them yourself, and ssw prove re-runs every claim's test. CI runs those checks on Linux, macOS and Windows.
Repo: https://github.com/sarthak-1111/stepswitch Feedback welcome, especially from anyone running agents on local models.