Choices, Choices: Notes From My Local-LLM Setup

A hands-on account of running local language models with coding agents reports that Anthropic's Claude Code auto mode, which sends each command to a model for a safety check, often times out when the checking model is a heavy local model that answers slowly. The author's fix is to point the permission-check slot at a faster mixture-of-experts model, Qwen3.6-35B-A3B-oQ8e-mtp, while Qwen3.8-27B-oQ4e-mtp handles main work, launched via omlx-cli flags because the tool ignores the usual environment variables. The post also recommends isolating unattended agents with Tart, the Apple Silicon VM tool whose repo now sits at github.com/openai/tart after Cirrus Labs joined OpenAI in April 2026, or Strands Box, which enforces policies written in AWS's Dogwood policy language.

Last time https://www.thoughtfultechnologist.com/p/hi-local-models?r=6ikndf I started testing things with local models - running different coding harnesses, comparing them and sharing the journey. Below is summary of some learnings after that: isolation, orchestration, one annoying timeout trap, and the choices I still haven’t made. Isolation first Before you let an agent run unattended, give it a box it can’t break out of. Two tools I liked: Tart https://tart.run/ starts macOS and Linux VMs on Apple Silicon, and it’s pretty cool. Cirrus Labs, the company behind it, joined OpenAI in April 2026 https://cirruslabs.org/ , and the repo now lives at github.com/openai/tart https://github.com/openai/tart . The good news: they said Tart would move to a more permissive open-source license, with no licensing fees. The open question is maintenance, since the original team moved on to other work at OpenAI. Development has continued in the new repo so far. I hope it stays that way. Strands Box https://github.com/strands-agents/box gives your agents isolation plus semantic policies written in Dogwood https://dogwood-policy.github.io/dogwood/ , AWS’s open-source policy language. You decide exactly which files, programs and hosts the agent can reach. Dogwood rules can also depend on what the agent has already done in the session, not just the current request. Policies are hard to write by hand, but Box ships a policy-authoring skill, so your coding agent can write them for you. With either of these in place, running your agent in YOLO mode stops being reckless. That matters for the timeout trap below. Who’s the orchestrator? I thought of two ways to mix a local model with a frontier one. They’re mirror images of each other. Option A: Claude Code is the boss. Run a coding harness on your local model I used opencode https://opencode.ai/ and give it to Claude Code as a tool. Then add a memory entry telling Claude Code to hand easier coding tasks to opencode. Claude plans and reviews; the local model does the grunt work, and you save tokens. Sounds good on paper, but in reality Claude Code starts acting like a bad manager - reviewing every single line produced and spending even more tokens. So this is not just a trivial “use opencode whenever you can skill”. I think there will be benefits on larger tasks, but I think it would require more steering as to what is created by the local model and what is done by claude code, which brings us to Option B: code is the boss. Flip it around. Write an orchestrator with something like AWS Strands https://strandsagents.com/ that is pure code, with no LLM making the decisions. Claude Code, opencode or anything else becomes a tool it calls to implement a task. More moving parts, more code to handle, more control - as usual. The auto-mode timeout trap Claude Code’s auto mode https://www.anthropic.com/news/claude-code-auto-mode sends a request to a model to check whether each command is safe to run. On a heavy local model that answers slowly, that check often times out. Claude Code doesn’t wait long enough, so it gives up on the action. You have two fixes: 1. Run in YOLO mode and handle safety elsewhere, with Tart or Strands Box see above setting the permissions. 2. Give the safety check its own, lighter model. Here’s how I do it with omlx: omlx-cli launch claude \ --haiku Qwen3.6-35B-A3B-oQ8e-mtp \ --opus Qwen3.8-27B-oQ4e-mtp \ --sonnet Qwen3.8-27B-oQ4e-mtp \ --debug-file cc-debug.log Qwen3.8-27B-oQ4e-mtp is the main model. The --haiku slot points at Qwen3.6-35B-A3B-oQ8e-mtp , a faster mixture-of-experts model, and that’s the one the permission checks use. Two notes: - omlx-cli ignores the usual environment variables, so you have to pass the model names as flags. - Keep --debug-file . It makes it easy for you or your agent to check that the flags were respected and the right model answered each call. Still unsettled Honestly, I haven’t settled on a local setup yet. - opencode or Claude Code as the main coding or not only harness? - How do I and shall I? share skills between them? - Should I automate my use cases as scripts, and then maintain those scripts? or just develop skills and subagents that do the job? - Or shall I build my own agents with a Strands orchestrator instead? - I’m starting to think what is it that makes a difference in a setup independently of the tool. I think that’s “my knowledge” and “my tools and skills” - so I need to work towards a way to make that portable found this after writing the article - https://ai-disclosures.org/ompi - Regarding “my knowledge” - use harness with .md files? Notion MCP? Obsidian with an LLM plugin? And which plugin?.. - How to make working with different models and tools locally more comfortable OpenUI? some other markdown-with-llm-capabilities local tool? Choices, choices, choices… What does your local setup look like? I’d love to steal a good idea or two. :