AIArticle
Local routers like Weave and 9Router cut API bills and bypass the context re-read tax.
AI coding assistants are incredibly capable, but they are also absolute token hogs. If you run terminal-based agents like Claude Code or use advanced environments like Cursor and Codex for hours a day, you have probably watched your API bills spike.
The problem is twofold. First, these tools default to sending every single request, no matter how trivial, to premium frontier models. Second, they suffer from what developers call the re-read tax. Every time an agent runs a terminal command, the output is dumped into the context window. On every subsequent turn of the conversation, the agent re-reads that entire history, compounding your token costs exponentially.
To fight back, developers are turning to a new class of local proxy routers. Tools like Weave's open-source router, 9Router, and Claude Code Router sit directly on your machine, intercepting API calls and routing them to the most cost-effective model that can handle the task. It is a genuine shift in how we manage local AI workflows, and it is quickly becoming a mandatory layer of the developer stack.
The Mechanics of Local Routing #
Instead of pointing your coding tools directly to Anthropic or OpenAI, a local router acts as a loopback proxy. It binds to a local port (such as localhost:8080
for Weave's router or localhost:20128
for 9Router) and exposes an OpenAI- or Anthropic-compatible endpoint.
When your agent sends a prompt, the proxy evaluates it and decides where to route it. Weave's router does this in under 50 milliseconds using a tiny, on-box embedder and a cluster scorer derived from Avengers-Pro. It avoids the slow, expensive vibes-based prompting that other routers use to categorize requests.
This architecture allows you to mix and match providers transparently. You can route heavy reasoning tasks to Claude 3.5 Sonnet, simple file lookups to DeepSeek-V3 via OpenRouter, and highly sensitive or repetitive tasks to a local Ollama instance running on your own hardware.
Bypassing the Re-Read Tax #
The financial killer in agentic workflows is not the initial prompt. It is the terminal output.
Consider a scenario where Claude Code runs a command like git diff
or a test suite that generates 2,000 tokens of output. Because of how agent loops work, those 2,000 tokens remain in the context window. Over a 10-turn conversation, the agent re-reads those same 2,000 tokens on every single turn, racking up 20,000 tokens of input usage from a single command.
To solve this, modern routers are integrating token compression engines like RTK (Rust Token Killer). RTK intercepts CLI command outputs and applies a multi-stage optimization pipeline:
Smart Filtering: Strips out ANSI escape codes, progress bars, and spinner artifacts.Grouping: Consolidates repetitive output lines.Deduplication: Collapses repeating patterns, such as long lists of passing tests.Truncation: Trims verbose success messages while preserving critical errors and warnings.
By compressing a 2,000-token diff down to 800 tokens, you save 12,000 tokens over that same 10-turn conversation. Multiply that across dozens of commands in a single coding session, and the savings easily reach 40% to 70%.
Setting Up a Local Router #
Adopting a local router is straightforward because they are designed as drop-in replacements. For example, Weave's router can be initialized with a single command:
npx @workweave/router --claude
This installer automatically detects your environment, grabs a router key, and configures Claude Code's settings. If you are using Codex, you can target it specifically:
npx @workweave/router --codex
This patches your ~/.codex/config.toml
file, adding a managed [model_providers.weave]
block and setting the active provider to weave
. Your existing OpenAI API keys flow through the router transparently.
For Cursor, the integration is handled via the editor's settings. You navigate to Settings → Models → Override OpenAI Base URL, point it to http://localhost:8080/v1
, and paste your local router key as the API key.
If you prefer to self-host the entire stack, including a local Postgres database and an observability dashboard, you can clone the repository and run the setup script:
echo "OPENROUTER_API_KEY=sk-or-v1-..." >> .env.local
make full-setup
This spins up the router at http://localhost:8080
and launches a local web UI where you can track routing decisions, latency, and token usage.
The Trade-offs of Auto-Routing #
While the cost savings are undeniable, local routing introduces several technical trade-offs that you must manage.
First, there is the issue of model consistency. If a router switches models mid-session (for example, falling back from Claude to Gemini or DeepSeek because of rate limits or cost rules), the agent's output style, system prompt adherence, and reasoning quality will change. For complex migrations or deep refactoring, this inconsistency can confuse the agent and lead to broken code. In those scenarios, it is safer to pin a specific model and disable automatic routing.
Second, integration methods vary by operating system. For token compression tools like RTK, Unix systems can use a clean hook mode that transparently rewrites commands before execution. Windows systems, however, cannot use hook mode and must fall back to modifying the CLAUDE.md
file. This instructs the agent to manually prefix every command with rtk
, which adds a small but persistent token overhead to every single turn.
Finally, you must be cautious with free API tiers and multi-account rotation. Automated coding agents can trigger rapid-fire requests that easily violate the terms of service of free providers, leading to sudden rate limits or account bans.
The Verdict #
Local AI routing is not hype. It is a highly practical solution to the very real problem of agent operating costs. If you only use an AI assistant occasionally for simple autocomplete, a local router is overkill. But if you are integrating agents deeply into your daily CLI and editor workflows, setting up a local proxy is the single most effective way to keep your API bills under control.
Sources & further reading #
Show HN: Smart model routing directly in Claude, Codex and Cursor— github.com - RTK, Model Routing, and the Community Tools That Actually Work With Claude Code - DEV Community— dev.to - How to integrate Cursor MCP with Codex | Composio— composio.dev - 9Router Tutorial: Connect Claude Code, Codex, and Cursor to One AI Router— knightli.com - Claude Code Router Guide 2026: Route Requests to Any AI Model | Get AI Perks— getaiperks.com
Rachel Goldstein· Dev Tools Editor
Rachel has been embedded in the developer tooling ecosystem for nearly eight years, covering everything from IDE wars and package-manager drama to the quiet rise of AI-assisted coding. She has a soft spot for open-source maintainers and an unhealthy number of terminal emulators installed on a single laptop.
Discussion 0 #
No comments yet
Be the first to weigh in.