Ornith-1.0 is a June 2026 open-weight coding model which claimed to have a genuinely different idea behind it: instead of learning inside a harness someone else built, it learns to build its own harness while it solves the task. The 9B model's benchmarks are strong for its size. I still couldn't get it running as a local Claude Code backend, and the reason why is more useful than the benchmark numbers.
Most coding models get trained inside a harness a human already built: a fixed prompt template, retry logic, a plan-then-act loop bolted on from the outside. Ornith claimed to do something different. During training, the model first proposes a plan for how to approach each task, then solves the task using that plan. Both the plan and the answer get scored, so the model learns to write better plans, not just better answers. Over time the two improve together.
On paper, the 9B model matches or beats Gemma4-31B, four times its size, on SWE-Bench Verified and Terminal-Bench 2.1. MIT-licensed, runs on a laptop.
Runtime: Ollama, model built from a raw HF GGUF (registry pull blocked by Zscaler on my network)
Proxy: LiteLLM, local instance, Anthropic-format /v1/messages front end
Client: Claude Code, pointed at the local proxy instead of api.anthropic.com
I've run this pattern before: point Claude Code at a local proxy instead of Anthropic, let the proxy translate to whatever model is actually running. Already wrote that up, with a cloud model behind it. This wasn't my first local model either: Gemma4, Qwen2, and Qwen3 before this. None stuck. The problem was never inference speed. It was trust: I don't have time to fact-check every answer for hallucination on top of doing the actual work.
Bug 1: silent template fallback. Building from a raw GGUF instead of ollama pull
ing the registry tag drops the chat-template metadata the manifest normally carries. Without it, Ollama falls back to bare prompt passthrough, no turn boundaries. Symptom: the model answers, keeps going, repeats the same answer, on a loop, because nothing tells it a turn ended. Fix: hand-write an explicit ChatML template plus a repeat penalty. Ollama applies neither by default when you skip the registry pull.
Here's the part of the Modelfile that actually fixed it, trimmed to the load-bearing pieces:
TEMPLATE """{{- if .Messages }}
{{- range .Messages }}
{{- if eq .Role "user" }}<|im_start|>user
{{ .Content }}<|im_end|>
{{ else if eq .Role "assistant" }}<|im_start|>assistant
{{ if .Content }}{{ .Content }}
{{- else if .ToolCalls }}
{{- range .ToolCalls }}<tool_call>
{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
</tool_call>
{{- end }}
{{- end }}<|im_end|>
{{ end }}
{{- end }}<|im_start|>assistant
{{ end }}"""
PARAMETER repeat_penalty 1.3
The <|im_start|>
/<|im_end|>
pairs are the turn boundaries the raw GGUF was missing. The <tool_call></tool_call>
XML tags are Hermes-style, matching what Ornith actually generates. That sets up the mismatch below: Ollama can render this template fine, but has nothing built in to parse a <tool_call>
block back into a real, executable tool call the way --tool-call-parser qwen3_xml
does under vLLM.
Bug 2: an unsupported capability request. Once output was clean, Claude Code didn't recognize the custom model ID and defaulted to requesting extended thinking on every call. Ollama rejected it outright: "does not support thinking"
, since the Modelfile never declared that capability. One env var fixed it:
export MAX_THINKING_TOKENS=0
The one that wasn't fixable in an afternoon: tool calls that were never real tool calls. First agentic prompt looked like it worked. Claude Code showed what looked exactly like a file-write tool call. The file never existed on disk. The model was generating text shaped like a tool call, not a structured one Ollama could parse and execute.
Root cause: Ornith is Qwen-derived, so it speaks Qwen's Hermes-style XML tool-call format. vLLM handles this with one flag: --tool-call-parser qwen3_xml
. Ollama has no equivalent. The tool-schema injection and the parse-back-into-a-real-call logic have to be hand-written into the Modelfile template.
This isn't unique to a manual GGUF build. A GitHub issue on ollama/ollama shows the same raw, unparsed tool-call XML on the official registry tag. Other write-ups on running Ornith locally list this as the most common complaint about the model on Ollama.
A model that confidently emits the exact right shape of a tool call, then does nothing, is much harder to catch than an outright error. If I hadn't checked the filesystem directly, I'd have assumed the loop was working.
The tool-call mismatch is real and documented by other Ornith-on-Ollama users, not just my setup. The manual GGUF build doesn't look like the deciding factor, since the linked issue hits the same wall on the official tag.
Not chasing this urgently. The pattern I rely on, Claude Code through a translating proxy instead of Anthropic directly, already runs day to day on a cloud model. This was the next test on top of that: same client, same proxy, swap the destination for something fully on-device. Didn't work this round. Same lesson a monitoring-agent build I wrote up earlier reached from a different angle: the wiring is rarely the hard part. The hard part is deciding how much to trust what's on the other end.
If a solid local model shows up that I can use and trust to do tasks such as write a file to disk, I'll test it properly and report back.