# Most AI Work Can Wait > Source: > Published: 2026-07-01 00:00:00+00:00 Most teams building agents pick the model first & the architecture second. That is backwards. The model choice is the last decision, not the first. What matters is the router, a small piece of code that decides which tier of model handles each request. Get the router right & 70-80% of traffic runs on local models that cost nothing per call, or on async models 1 that reduce AI spend by 90%+. Brian Armstrong made the same point last week about how Coinbase cut AI spend in half while token usage grew 2, paraphrasing : How to keep AI spend flat while token usage grows exponentially : not with friction & spend alerts. With better defaults, routing, & caching. Engineers can choose any model they want, but defaults matter. The routing problem has three layers, and each does a distinct job : **Skill classifier** turns a raw user request into a concrete operation. It answers what the task is. Draft-a-reply, summarize-a-repo, run-a-migration. The classifier is intent recognition.**Router** decides which tier executes the classified operation. It answers which model runs it. The router does not read the prompt. It reads the classifier’s label plus a few features : complexity, context size, historical success rate.**Model selector** picks the cheapest model within a tier that meets a confidence threshold. Classifier & router are not the same. The classifier is a language problem ; the router is a scheduling problem. Conflating them buries the model choice inside the prompt & kills the ability to A/B different models against the same operation. Local compute is close to free. Async batch reasoning runs two orders of magnitude cheaper than real-time inference 1. So the real question is narrower : what fraction of work needs real-time answers? Surprisingly little, once the system can queue work. Queueing is why this works. A draft reply, a repo summary, a diligence memo, a nightly evaluator run : none of these need to return in a second. We built the first version of this into our agent runtime. The router already scored tasks on complexity, context size, & local memory retrieval. Two feedback mechanisms now sit on top of the router, & they operate on different time scales : **Synchronous failure-mode signals.** A predictor annotates each incoming route with five features : missing repo context, long dependency chains, risky migrations, security-sensitive prompts, & high-consequence writes.**Nightly closed-loop feedback.** A batch evaluator scores yesterday’s traces overnight & updates the router’s weights, running on[async inference on Sail](/sail-inference-queue/)to keep the evaluation cost near zero. The synchronous predictor catches known-hard tasks before they fail. The nightly loop discovers new failure modes the predictor missed. Once skill distillation flattens the operation set, 70-80% of agent traffic can run on local models 3 for most non-coding work. The implication : design your system around routing, not around models. Pick your models last. - [Full Sail on Asynchronous Inference](/sail-inference-queue/)— the cost delta between real-time & async batch inference.[↩︎](#fnref:1)[↩︎](#fnref1:1) - [Brian Armstrong on X](https://x.com/brian_armstrong/status/2070670644577280109)— Coinbase cut AI spend nearly in half while token usage grew, via better defaults, routing, & caching.[↩︎](#fnref:2) - [Skill Distillation](/the-pi-agent-skill-distillation/),[Teaching Local Models to Call Tools Like Claude](/distilling-claude-into-local-models/), &[The Minimill of AI](/using-local-ai-to-work-faster/).[↩︎](#fnref:3)