A tiny local router model picks the cheapest capable AI model for each coding task, cutting cost while nearly matching perfect routing in tests.
What is Panda Orchestrator? #
Panda Orchestrator is a system built around a small local model, called Panda Router, that decides which AI model should handle a given coding task. Instead of sending every task to one expensive model or guessing with a cheap one, the router reads the task, scores it against a pool of available models, and picks the one most likely to succeed at the lowest cost. It was built as part of Panda Code, an open-source workspace for running Claude Code, Codex, Kimi, Open Code, and other coding agents side by side.
TL;DR #
- Panda Router is a 32-million-parameter decision model , distilled from a 150-million-parameter teacher trained on Modal, quantized to int8, and exported to ONNX at roughly 31 MB, small enough to run on a CPU in 10 to 25 milliseconds with no GPU required.
- The router doesn’t write code or judge quality itself : it outputs a task category, a difficulty score from 1 to 10, required skills, and flags like tool or image needs, then a separate rules-based scoring system (not a neural network) matches that profile against model cards for cost, speed, and chance of success.
- Training leaned on cheap synthetic labeling plus real outcome data : DeepSeek V4.1 Flash labeled about 8,000 tasks for $2.77, while an “attempt and grade” process had real models actually try tasks and a strict judge score pass or fail, until credits ran out around $44 to $50 total.
- The escalation ladder is the core trick : each task starts on the cheapest model likely to succeed, and if it fails a check, the task moves to a strictly stronger model next, never down, capped at two retries.
- Benchmark results on 150 held-out tasks with a nine-model pool showed single-pick routing matching an always-use-the-best-model baseline at lower cost , and escalation with up to two retries pushed success from about 77-79 out of 100 up to roughly 91, close to the theoretical best case of 94.
- The whole project cost around $50 in compute and training and went from an empty repository to a working version in about three days, according to its creator.
- The router is now built directly into Panda Code in Rust with an embedded ONNX runtime , so it runs with no Python dependency, and it’s also available standalone as an MIT-licensed CLI, HTTP server, Python API, and MCP server for use with other agent frameworks.
regulatory compliance and security.
- Name
- Maria Alvarez
- Date of birth
- SSN
- Diagnosis
- Balance due
- $412.80
How does the router actually decide which model to use? #
The system splits judgment and policy into two separate layers on purpose. The neural network only produces a profile of the task: its category, a 1-10 difficulty rating, the skills it needs, and flags for things like tool use or image input. A conventional piece of code then takes that profile and scores every available model against three factors: estimated chance of success, expected cost, and speed.
Keeping the scoring logic in plain code rather than inside the model matters for debugging. If the router makes an odd choice, the rules can be inspected and adjusted directly instead of requiring retraining. Every model in the pool is represented by a small JSON card containing skill ratings, price, context window, and latency. When a new model launches, which now happens almost constantly, adding a card is enough. No retraining is needed for the router to account for it.
How was Panda Router trained? #
The profiler architecture is based on Etin, a family of small MIT-licensed encoder models trained on open data that includes code, with all sizes sharing one tokenizer. That let the builder train a larger 150-million-parameter teacher model on Modal and distill it down into the 32-million-parameter student that ships in production.
Getting training data was the harder problem. Labeling roughly 8,000 tasks with DeepSeek V4.1 Flash cost under $3, but those labels only reflect guesses about difficulty, not real outcomes. To get actual outcome data, the process called “attempt and grade” had real models try tasks for real, with a strict judge grading each attempt as pass or fail. That data is what let the system learn which models genuinely succeed at which kinds of tasks, not just which ones look capable on paper.
From those real outcomes, a separate “success head” was trained to output a calibrated probability of success per model, plus a lower-confidence bound. The router makes its decisions using that lower bound rather than the optimistic estimate, so it avoids betting on a model that only looks good in aggregate. Cost comparisons also factor in the cost of failed attempts, since a cheap model that fails half the time isn’t actually cheap once retries are counted. Two training details stand out. The difficulty-rating loss function was deliberately lopsided: if the model has to err, it’s tuned to overestimate difficulty rather than underestimate it, since underrating a task is the costlier mistake. Separately, the router was fine-tuned on about 1,500 deliberately vague prompts (“make the app better” with no scope), which are rated as harder tasks since any model struggles with ambiguous requirements.
What results did it actually produce? #
1,284
37
6
| Vendor | Tier | Risk | Approval <sup>NEW</sup> |
|---|---|---|---|
| Northwind Supply | Tier 1 | Low | Approved |
| Acme Logistics | Tier 2 | Medium | Approve |
| Helios Data | Tier 1 | High | Pending |
| Granite Freight | Tier 3 | Low | Approved |
| Pinecrest Labs | Tier 2 | Low | Approve |
Testing used 150 held-out tasks across a nine-model pool. Always routing everything to one strong model solved about 77 to 79 out of 100 tasks for roughly $1 per 100 tasks. A single pick from the router matched or slightly beat that success rate at lower cost. Once escalation was enabled, allowing up to two retries that always move to a stronger model after a failure, success climbed to about 91 out of 100. The theoretical ceiling, where the “correct” model is known in advance for every task, was 94 out of 100. That puts the router’s escalation-enabled performance within a few points of a best-case scenario that isn’t achievable in practice.
Two later versions (5 and 6) were trained with more data and a fresh evaluation set, but both missed pre-written pass criteria in at least one area, so version 4.3 remains the default. Testing version 6 also surfaced a grading bug: answers longer than 6,000 characters were being truncated before grading, which unfairly penalized a model that tends to write longer, more thorough responses.
How does this show up inside Panda Code? #
Inside Panda Code, Panda Router is built into the “agent teams” feature, where a lead architect agent splits a task into assignments for parallel worker agents. Previously, the architect had to guess which model each worker should use, often defaulting to the most powerful model for trivial work or underpowered models for hard work.
Now, clicking the team button offers four modes: no team, cheap team, balanced, and best team. Cheap and balanced modes start each worker on the cheapest model likely to clear a minimum success threshold. If a worker’s output fails a check (tests, type checks, or review), the architect swaps it for a stronger model per the escalation ladder, capped at two retries before it reports failure rather than looping indefinitely.
The system also added failover handling: if a worker fails because of an infrastructure problem (rate limit, crashed CLI, expired login) rather than a bad model choice, Panda Code retries the same model through a different installed harness. A model health system s models that error out, starting at 10 minutes and doubling up to 2 hours, and those d models are skipped by the router entirely until they recover.
Users can also override the router’s defaults by building a manual tier list (S through D) for their installed models, which blends 70% user ranking with 30% the router’s own score at each difficulty level, directly influencing both model choice and the order of the escalation ladder.
Is this kind of model routing worth using? #
For anyone running multiple coding agents or paying per-token for a mix of frontier and budget models, automated routing addresses a real cost problem: strong models are often wasted on trivial work, while weak models fail silently on hard work and create cleanup costs. The benchmark gap between “always use the expensive model” and “router plus escalation” (77-79 versus 91 out of 100, at lower average cost) suggests the approach captures most of the benefit of perfect routing without needing to know in advance which model fits which task. The creator is upfront about limitations: many model cards are still estimates rather than measured benchmarks, the success rates reflect a strict judge rather than real-world leniency, and Intel Mac and x86 Linux currently fall back to simple keyword matching because no compatible ONNX runtime build exists yet for those platforms. The system’s escalation logic is also only as reliable as the pass/fail check it relies on, so routing quality depends on having real tests or review steps in place.
Frequently Asked Questions #
What is Panda Router exactly?
Without shadow IT.
Every app, approved
✓
✓
✓
✓
✓
It’s a small, locally run decision model (around 32 million parameters, quantized and exported to ONNX) that reads a coding task and outputs a profile (category, difficulty, required skills, flags) used to pick the most cost-effective capable model for that task.
Does Panda Router write or review code itself?
No. It only produces a task profile. A separate rules-based scoring system uses that profile, along with model cards for cost, speed, and success probability, to select a model. Code quality checks (tests, type checks, review) are handled elsewhere in the pipeline.
How much did it cost to build?
The full project, including a modal training run for the teacher model and distillation into the student model, cost around $50 total, with labeling costs under $3 and attempt-and-grade evaluation running until open router and DeepSeek credits were exhausted around $44.
What is the escalation ladder?
It’s an ordered list of models from cheapest to strongest for a given task. Work starts on the cheapest model likely to succeed; if it fails a check, the task moves to the next stronger model on the list, and it never moves back down after a failure.
Can this routing approach be used outside Panda Code?
Yes. The underlying router is released under an MIT license with a CLI, HTTP server, Python API, and MCP server, making it usable with other coding agent setups beyond the Panda Code application it was built for.