# Choices, Choices: Notes From My Local-LLM Setup

> Source: <https://www.thoughtfultechnologist.com/p/choices-choices-notes-from-my-local>
> Published: 2026-11-14 09:55:49+00:00

[Last time](https://www.thoughtfultechnologist.com/p/hi-local-models?r=6ikndf) I started testing things with local models - running different coding harnesses, comparing them and sharing the journey.

Below is summary of some learnings after that: isolation, orchestration, one annoying timeout trap, and the choices I still haven’t made.

## Isolation first

Before you let an agent run unattended, give it a box it can’t break out of. Two tools I liked:

**[Tart](https://tart.run/)** starts macOS and Linux VMs on Apple Silicon, and it’s pretty cool. Cirrus Labs, the company behind it, [joined OpenAI in April 2026](https://cirruslabs.org/), and the repo now lives at [github.com/openai/tart](https://github.com/openai/tart). The good news: they said Tart would move to a *more* permissive open-source license, with no licensing fees. The open question is maintenance, since the original team moved on to other work at OpenAI. Development has continued in the new repo so far. I hope it stays that way.

**[Strands Box](https://github.com/strands-agents/box)** gives your agents isolation plus semantic policies written in [Dogwood](https://dogwood-policy.github.io/dogwood/), AWS’s open-source policy language. You decide exactly which files, programs and hosts the agent can reach. Dogwood rules can also depend on what the agent has *already done* in the session, not just the current request. Policies are hard to write by hand, but Box ships a policy-authoring skill, so your coding agent can write them for you.

With either of these in place, running your agent in YOLO mode stops being reckless. That matters for the timeout trap below.

## Who’s the orchestrator?

I thought of two ways to mix a local model with a frontier one. They’re mirror images of each other.

**Option A: Claude Code is the boss.** Run a coding harness on your local model (I used [opencode](https://opencode.ai/)) and give it to Claude Code as a tool. Then add a memory entry telling Claude Code to hand easier coding tasks to opencode. Claude plans and reviews; the local model does the grunt work, and you save tokens.

Sounds good on paper, but in reality Claude Code starts acting like a bad manager - reviewing every single line produced and spending even more tokens. So this is not just a trivial “use opencode whenever you can skill”. I think there will be benefits on larger tasks, but I think it would require more steering as to what is created by the local model and what is done by claude code, which brings us to

**Option B: code is the boss.** Flip it around. Write an orchestrator with something like [AWS Strands](https://strandsagents.com/) that is pure code, with no LLM making the decisions. Claude Code, opencode or anything else becomes a tool it calls to implement a task. More moving parts, more code to handle, more control - as usual.

## The auto-mode timeout trap

Claude Code’s [auto mode](https://www.anthropic.com/news/claude-code-auto-mode) sends a request to a model to check whether each command is safe to run. On a heavy local model that answers slowly, that check often times out. Claude Code doesn’t wait long enough, so it gives up on the action.

You have two fixes:

1. **Run in YOLO mode** and handle safety elsewhere, with Tart or Strands Box (see above) setting the permissions.
2. **Give the safety check its own, lighter model.** Here’s how I do it with omlx:

```
omlx-cli launch claude \
  --haiku Qwen3.6-35B-A3B-oQ8e-mtp \
  --opus Qwen3.8-27B-oQ4e-mtp \
  --sonnet Qwen3.8-27B-oQ4e-mtp \
  --debug-file cc-debug.log
```

`Qwen3.8-27B-oQ4e-mtp` is the main model. The `--haiku` slot points at `Qwen3.6-35B-A3B-oQ8e-mtp`, a faster mixture-of-experts model, and that’s the one the permission checks use.

Two notes:

- omlx-cli ignores the usual environment variables, so you have to pass the model names as flags.
- Keep `--debug-file` . It makes it easy for you (or your agent) to check that the flags were respected and the right model answered each call.

## Still unsettled

Honestly, I haven’t settled on a local setup yet.

- opencode or Claude Code as the main coding(or not only) harness?
- How do I (and shall I?) share skills between them?
- Should I automate my use cases as scripts, and then maintain those scripts? or just develop skills and subagents that do the job?
- Or shall I build my own agents with a Strands orchestrator instead?
- **I’m starting to think what is it that makes a difference in a setup** ***independently*** **of the tool. I think that’s “my knowledge” and “my tools and skills” - so I need to work towards a way to make that portable** (found this after writing the article - https://ai-disclosures.org/ompi)
- Regarding “my knowledge” - use harness with .md files? Notion MCP? Obsidian with an LLM plugin? And which plugin?..
- How to make working with different models and tools locally more comfortable OpenUI? some other markdown-with-llm-capabilities local tool?

Choices, choices, choices…

**What does your local setup look like? I’d love to steal a good idea or two. :)**