# Building Maestro AI: Routing LLM Calls So Your Agent Doesn't Burn Sonnet on Summaries

> Source: <https://dev.to/david_shibley/building-maestro-ai-routing-llm-calls-so-your-agent-doesnt-burn-sonnet-on-summaries-1m36>
> Published: 2026-07-12 16:19:08+00:00

*How I built a harness-agnostic model router for Cursor and Claude Code and what broke along the way.*

When you use Claude Sonnet for everything in Cursor, you pay premium prices for work a local Llama could handle in two seconds. When you use only Ollama, you get worse results on architecture reviews and multi-step refactors.

I wanted a middle layer: **cheap by default, escalate when the task actually needs it.** Not a new chatbot, a dispatcher the existing agent could call for subtasks.

That layer became ** Maestro AI**.

Think of your coding agent as a conductor. It should keep architecture, multi-file edits, and complex reasoning. Maestro is the section leader who picks which musician plays each part:

The conductor doesn't stop conducting. It delegates the small solos.

Cursor picks one model per chat. Agent sessions spawn dozens of implicit subtasks; summarize this file, rewrite this message, extract these fields, format this JSON. The harness doesn't automatically route those to cheaper models.

Maestro adds **per-call routing** with:

We classify prompts with deterministic rules: keywords, patterns, tool presence, context size. Examples:

`"summarize this README"`

→ `summarization`

, easy, low risk → `local_strong`

`"Design system architecture for event sourcing"`

→ `architecture`

, hard → `premium`

`"make me an HTML demo page"`

→ `code_edit`

, easy → `local_fast`

Early versions had painful false positives. The word **"design"** in "design a simple HTML landing page" triggered architecture routing and sent demo pages to Sonnet. We fixed that with bounded patterns: `isSimpleUiTask()`

, `isDemoShowcaseTask()`

, and stricter `SYSTEM_ARCHITECTURE_SIGNALS`

vs casual "design" usage.

Another gotcha: agents sent **meta-prompts** to the router, *"Determine routing for building a demonstration of model-router capabilities"*, instead of the literal task. That hit architecture keywords again. MCP tool descriptions now explicitly say: pass the literal task, not a routing meta-description.

Routing is a decision tree:

```
hard / high-risk / tool+code / long context / architecture → premium
medium coding / debugging / refactoring → hosted_oss
summarization / rewriting / extraction → local_strong
easy / formatting / simple UI → local_fast
```

Then overlays:

`max_tier`

`budget_usd`

`always_prefer_local`

Our stack:

`:11434`

- llama3.2, qwen3:8b`:4000`

- proxies to Featherless (GLM, Qwen Coder) and Bedrock (Sonnet)Config shape:

```
"local_strong": {
  "primary": { "provider": "litellm", "model": "glm", "baseUrl": "http://localhost:4000/v1" },
  "fallback": { "provider": "ollama", "model": "qwen3:8b", "baseUrl": "http://localhost:11434/v1" }
}
```

When LiteLLM dies, `local_strong`

stays on-tier via Ollama; it doesn't jump to premium or drop to tiny Llama for a summarization task.

We learned this the hard way: a **zombie LiteLLM process** existed but wasn't listening on `:4000`

. Routing failed mysteriously until we built `maestro doctor`

; process check, port check, `/v1/models`

, `FEATHERLESS_API_KEY`

, per-tier endpoint probes.

`routedLLMCall()`

doesn't fire-and-forget. After each call:

`completion_tokens > 0`

but no visible text? Fail and escalate.If checks fail → retry same tier (once) → escalate to next tier → log telemetry.

We hit a real bug here: GLM returned an **empty string after 130 seconds**, but `non_empty`

passed. Root cause cluster:

`trim().length > 0`

let through `\u0000`

and zero-width characters`reasoning_content`

or array parts we didn't parse`ProviderError("empty")`

didn't trigger escalationFix: shared `content-extract.ts`

, `hasMeaningfulContent()`

, explicit empty escalation, `content_integrity`

check for token-without-text.

Agents don't shell out reliably. Maestro exposes MCP tools:

| Tool | Purpose |
|---|---|
`maestro_route` |
Dry-run routing; no LLM call |
`maestro_ask` |
Route + execute + auto-escalate |
`maestro_doctor` |
Infrastructure diagnostics |
`maestro_stats` |
Telemetry dashboard |
`maestro_feedback` |
Thumbs up/down for tuning |

Every route/ask response includes a **full routing report**: analysis, debug trace, probe status, fallback reason. We learned that hiding debug behind `debug: true`

cost two debugging rounds; visibility is now always on.

Pass once per session:

```
{
  "session_id": "cursor-main",
  "budget_usd": 0.50,
  "max_tier": "hosted_oss",
  "always_prefer_local": true
}
```

Budget is **enforced** - telemetry sums spend per `session_id`

, caps tier selection, blocks escalation when exhausted.

Maestro is published on npm as [ maestro-ai](https://www.npmjs.com/package/maestro-ai). You don't need to clone the repo — the package ships the compiled CLI, MCP server, and bundled config profiles.

```
npm install -g maestro-ai

maestro init --profile ollama-only   # or default / cloud-only
ollama pull llama3.2:latest
ollama pull qwen3:8b
maestro doctor
maestro route "summarize this paragraph" --debug
```

Two binaries are exposed:

| Command | What it runs |
|---|---|
`maestro` |
CLI — `init` , `route` , `ask` , `doctor` , `stats` , … |
`maestro-mcp` |
MCP server for Cursor / Claude Code |

`npm install`

runs `prepare`

, which builds `dist/`

from TypeScript — consumers get a ready-to-run package, not raw source.

```
npx maestro-ai init --profile ollama-only
npx maestro route "rewrite this commit message" --debug
npx maestro-mcp   # run MCP server once (stdio)
```

`npx`

downloads the package on first use and caches it. Good for trying Maestro or CI scripts that need a single routed call.

After `maestro init`

, check `~/.maestro-ai/mcp-config.json`

. For npm installs it typically looks like:

```
{
  "mcpServers": {
    "maestro-ai": {
      "command": "npx",
      "args": ["-y", "maestro-mcp"],
      "env": {
        "MAESTRO_CONFIG": "/Users/you/.maestro-ai/config.json",
        "LITELLM_MASTER_KEY": "sk-litellm-local"
      }
    }
  }
}
```

No hardcoded clone paths — `npx maestro-mcp`

resolves the binary from the published package. Merge that block into Cursor → Settings → MCP and reload.

```
npm install maestro-ai
js
import { routedLLMCall, dryRunRoute } from "maestro-ai";

const preview = await dryRunRoute({
  messages: [{ role: "user", content: "Extract dates from this email." }],
});

console.log(preview.routing.tier, preview.routing.model);

const result = await routedLLMCall({
  messages: [{ role: "user", content: "Extract dates from this email." }],
  overrides: {
    session: { sessionId: "my-app", maxTier: "hosted_oss", budgetUsd: 0.25 },
  },
});

console.log(result.response.content);
console.log(result.telemetryId);
```

Config and telemetry still live under `~/.maestro-ai/`

(created by `maestro init`

). Override with `MAESTRO_CONFIG`

if you want a project-local config file.

npm |
git clone |
|
|---|---|---|
| Best for | Using Maestro, MCP, CLI | Contributing, patching routing rules |
| Install | `npm install -g maestro-ai` |
`git clone` + `npm install`
|
| MCP | `npx maestro-mcp` |
`node dist/mcp-server.js` in clone |
| Updates | `npm update -g maestro-ai` |
`git pull` + `npm run build`
|

We dogfood the npm package locally too: `npm pack`

produces a tarball you can install with `npm install -g ./maestro-ai-0.4.1.tgz`

to verify the published artifact before release.

Personal hardcoded paths don't scale. `maestro init`

:

`~/.maestro-ai/`

`default`

, `ollama-only`

, `cloud-only`

)`npx maestro-mcp`

when npm-installed)`ollama pull`

models`.env.example`

Colleagues can run `maestro init --profile ollama-only`

with **no LiteLLM, no cloud keys**; just Ollama.

Rules stay the floor; ML can sit on top later.

| Choice | Why |
|---|---|
| TypeScript standalone package | Fits Cursor MCP, CLI, npm, any Node harness |
Raw `fetch` to `/v1/chat/completions`
|
Ollama and LiteLLM are OpenAI-compatible |
| JSONL telemetry | Simple, grep-friendly, no DB |
| Vitest | Fast, 77 tests covering routing edge cases |
| MCP over custom HTTP | Agents already speak MCP natively |

No LangChain. The router is ~27 source files; you can read the whole thing in an afternoon.

What works well:

`maestro doctor`

catching zombie processes before routing failsWhat's still manual:

```
git clone https://github.com/David-J-Shibley/maestro-ai.git
cd maestro-ai && npm install
maestro init --profile ollama-only
ollama pull llama3.2:latest && ollama pull qwen3:8b
maestro doctor
maestro route "summarize this paragraph" --debug
```

Merge `~/.maestro-ai/mcp-config.json`

into Cursor MCP settings. In chat, the agent can call `maestro_ask`

for cheap subtasks while keeping hard work in its own session.

`maestro init`

mattered more than another routing featureHarness wiring (Benchy, Claude Code hooks), feedback-driven stats, premium pool rotation, and eventually learned routing once telemetry has enough rows.

*Maestro AI is open source (MIT): github.com/David-J-Shibley/maestro-ai*