{"slug": "the-router-inside-kilos-auto-router", "title": "The Router inside Kilo’s Auto Router", "summary": "Kilo's Auto Router automatically selects the cheapest model that passes its Kilo Bench benchmarks for a given task, drawing on a gateway of over 500 models, according to a company writeup on how the router works. A lightweight classifier categorizes each prompt — code generation, architecture planning, migration, or tool use — then the router consults a routing table scored on pass/fail accuracy and average cost, with tiers including Auto Frontier, Auto Free, and Auto Efficient. Teams can build custom model pools, and the router re-benchmarks those models with the same deterministic pass/fail methodology rather than reusing platform scores.", "body_md": "Auto routing is the bleeding edge of using AI inference these days. The idea that each request could be automatically sent to the model with exactly the right balance of cost and performance is what we’re all shooting for.\n\nThat’s exactly what Kilo’s Auto Router does – you don’t switch models by hand, and you don’t guess which one is worth the money this week. Instead, it automatically chooses the cheapest model that meets our benchmarks for the task at hand.\n\nBut the coders using Kilo’s auto router are curious people and want to know what this actually means. So here’s a writeup on how auto-router works.\n\n## But first, why auto router?\n\nBefore we get into the how, it’s worth pausing for a moment at why to auto route. If you only had one model, there’d be no need to choose the right model for any given task. The model you have is always the right one.\n\nBut Kilo’s Gateway has over 500 models in it. What we’ve observed is that there is indeed a select, “frontier” group of models that are the most powerful - such as Fable, Astra, and Opus. But these models are also exponentially more expensive than another set that can perform quite well. A number of open-weights and older models can perform extremely well on targeted, day-to-day tasks at a fraction of the cost.\n\nAt Kilo, we run our own Kilo Bench benchmarks that are captured by our [leaderboard](http://kilo.ai/leaderboard). Each benchmark hands a model a task and checks the output against a known-good result, so the model either passes or it fails. There’s no judge model scoring the answer on a curve; it’s a deterministic pass/fail.\n\nThese benchmarks attempt to capture real-world performance on coding tasks. We run them against a variety of models, and we’ve consistently found that there’s a category of model that has strong performance with much lower cost than Frontier models.\n\nIn the chart above, the models to the upper right “punch above their weight” in terms of performance. It’s *these* models that our Auto Balanced and auto efficient models route to.\n\n## **Routing to the right model**\n\nBehind Kilo Bench, we’re creating a routing table. When we execute the Kilo Bench runs, we classify the tasks so that for each model/task combination, we can see how often it succeeds and how much it costs on average. It looks something like this:\n\nWhen you send a prompt to Auto Balanced, a lightweight classifier reads the request and decides what kind of task it is: code generation, architecture planning, a migration, tool use, or one of a handful of other categories.\n\nOur classifier is tuned for speed because a slow classifier would defeat the point of automatic routing, and we benchmark its accuracy separately. That’s because a misread sends the request to a table that is built for the wrong kind of work entirely.\n\nOnce the classifier settles on a task type, the router looks it up in the table. We scored every candidate model ahead of time on two things: how often it produces a correct answer for that kind of task, and what it costs to get there.\n\nThe model that wins depends on which tier you’re running. Auto Frontier picks the strongest model available regardless of price. Auto Free stays within the best models we offer at no cost. Auto Efficient cross-references your prompt against Kilo Bench and finds the least expensive model already proven proficient at that specific kind of task.\n\n## **Your pool, your table**\n\nThe default table covers a curated pool of models that we picked for their accuracy relative to their price, but teams aren’t stuck with our picks. You can adjust the pool if your organization has a contract with a specific inference provider, or a compliance requirement that rules out certain vendors entirely.\n\nWhen you build a custom pool the router doesn’t reuse the platform’s scores against your models; it benchmarks your models on the same pass/fail methodology and builds a table specific to your needs.\n\n## **Overriding the router**\n\nThe router makes a price-and-accuracy tradeoff on your behalf using numbers we computed ahead of time, but nothing about that stops you from overriding it. You can pick a model manually for any task, and the router steps aside entirely. Leave a tier of auto model running and it applies that same lookup, automatically, to every request you send through it, whether that’s one request a day or several thousand.\n\nYou don’t need a private jet to travel down the street. In the same vein, it no longer makes sense to use the most powerful AI model for every task, especially when there are other, cheaper options that are built especially for smaller, more refined tasks. That’s why a lot of people consider auto-routing to be the future of AI model selection, because software development is a dynamic process, and that should extend to the tools you use, and the prices you pay.", "url": "https://wpnews.pro/news/the-router-inside-kilos-auto-router", "canonical_source": "https://blog.kilo.ai/p/the-router-inside", "published_at": "2026-09-18 21:24:34+00:00", "updated_at": "2026-09-18 21:54:56.871626+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-products", "large-language-models", "ai-tools"], "entities": ["Kilo", "Kilo Auto Router", "Kilo Gateway", "Kilo Bench", "Auto Frontier", "Auto Free", "Auto Efficient", "Fable"], "alternates": {"html": "https://wpnews.pro/news/the-router-inside-kilos-auto-router", "markdown": "https://wpnews.pro/news/the-router-inside-kilos-auto-router.md", "text": "https://wpnews.pro/news/the-router-inside-kilos-auto-router.txt", "jsonld": "https://wpnews.pro/news/the-router-inside-kilos-auto-router.jsonld"}}