{"slug": "smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower-cost-per", "title": "Smart Routing in Unity AI Gateway: Match frontier quality with 30%+ lower cost per task", "summary": "Databricks launched Smart Routing in Unity AI Gateway, now in Beta, which automatically matches coding tasks to the most cost-effective model and harness, achieving 30%+ lower cost per task while matching frontier quality. On internal coding workloads, Smart Routing outperformed any single model at 65% of the cost per task of Opus 5, and on public benchmarks it matched Opus 5 at less than half the cost.", "body_md": "How we built task-aware routing across models and harnesses to cut costs on real-world coding tasks at Databricks\n\nby [Ankit Mathur](/blog/author/ankit-mathur), [Ivan Zhou](/blog/author/ivan-zhou), [Bryan Qiu](/blog/author/bryan-qiu), [Rohit Agrawal](/blog/author/rohit-agrawal-ai-gateway), [Elise Gonzales](/blog/author/elise-gonzales) and [Kelly Albano](/blog/author/kelly-albano)\n\nThe price and performance frontier for coding tasks features a huge diversity of models and harnesses: in 2026 alone, we’ve seen 33 new models released. In our prior post about [benchmarking against the Databricks codebase](https://www.databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase), we found that models cluster into capability tiers and that much everyday work (e.g., flipping a flag, a single-file edit, a well-scoped bug fix) did not require the most expensive models.\n\nSo how do you reduce AI coding costs without sacrificing developer productivity? One of the biggest opportunities is matching each task to the right model instead of defaulting every task to the most capable (and most expensive) option. [Just leveraging lower cost models can save you 50%+](https://www.databricks.com/blog/managing-ai-coding-costs-scale), but it’s incredibly daunting for users. With a proliferation of great models and capable harnesses, coding agent users are constantly faced with choice overload. Instead of wasting time trying to select the best model for every task, many are setting the most capable at the highest effort and moving on. Instead of asking users to choose or stunting productivity with hard caps, we knew we needed to innovate.\n\n**That’s why we’re launching the next major cost control in Unity AI Gateway: Smart Routing, now available in Beta.** [Unity AI Gateway](https://www.databricks.com/blog/unity-ai-gateway-generally-available) provides a central place to get access to AI, manage spend, and enforce controls across your entire enterprise, and Smart Routing adds intelligent optimization by automatically matching tasks to the right model based on complexity. Smart Routing works directly in Claude Code and Codex, allowing you to optimize the tools developers already use.\n\nAnd we’re going beyond model routing. With [Omnigent](https://www.databricks.com/blog/introducing-omnigent-meta-harness-combine-control-and-share-your-agents), our meta-harness for coding agents, teams can leverage the full power of Smart Routing by optimizing across both models and coding harnesses, giving developers the right combination for the task without having to choose it themselves.\n\nThe results speak for themselves: On internal coding workloads, Smart Routing outperformed any single model at just 65% of the cost per task of a leading model like Opus 5. On public benchmarks, Smart Routing matched Opus 5 on performance at *less than half the cost*.\n\nHere’s what we learned:\n\nLet’s walk through how we built this.\n\nIntelligent model routing selects the model best suited for a task based on factors like complexity, capability and cost. For coding agents, an important decision is when that routing should happen.\n\nThere are generally two approaches to routing:\n\nWe opted for task-aware routing to preserve cache efficiency while matching each coding task to the appropriate model and harness**.** The most interesting problem is judging how difficult a task is before starting it. We wanted to start simple, so our router currently uses a single policy and applies it to every task the same way.\n\nFirst, we classify the task. For this, we use a cheaper, low-latency model that reads the task description and labels it with a handful of semantic fields: what part of the system changes, what code evidence the prompt carries (a snippet, a traceback, or nothing explicit), how it appears to be failing, how localized the fix looks, and what kind of project it belongs to. From these, the router derives a task-type family and a language family. Using a frontier model would tax every request (even the simple ones we want to save on), so the extractor is intentionally small and fast.\n\nThen, we triangulate on which model class is best. The router defaults to a medium-sized model and uses the labels to move in either direction, escalating to a more expensive model when the task demands frontier-level capability and knowledge or delegating down to a cheaper one when it does not. This means the single policy can leverage a whole suite of models.\n\nThe early results are promising. Against our own internal benchmark, which no labs have had access to, we saw 35% savings. Against public coding benchmarks that demonstrate our results generalize, we achieved 56% cost savings. We expect to see this grow as we learn more about our own use cases and with our design partners.\n\nSmart Routing handles the routing decision, but then you need to be able to act on it. It works natively inside of Claude Code and Codex, but for coding agents, we see better performance by choosing not only the right model, but also the right coding harness. Helping engineers take advantage of the chosen model and harness requires a layer sitting above the individual coding sessions to orchestrate across them. This is why we built [Omnigent](https://www.databricks.com/blog/introducing-omnigent-meta-harness-combine-control-and-share-your-agents).\n\nSmart Routing is implemented in Omnigent at two levels:\n\nFirst, developers using Omnigent can select Smart Routing instead of manually choosing a specific coding harness. Omnigent then automatically selects both the harness and model for each task, with model routing powered by Smart Routing in Unity AI Gateway. This design gives developers and admins the freedom to provide customizations, such as org-level guidance or the option to use previous conversation history, without changing the client every time.\n\nThis also means all sub-agent launches go through the Smart Routing API, allowing sub-agents to leverage a different harness and model. The user's initial prompt is often underspecified and hard to judge in terms of complexity, so sub-agents allow you to adjust new work based on new information, with a fresh cache and clear instructions. A single task can experience nuanced routing decisions across planning and parallel sub-agent work (e.g., you can route large codebase summarization tasks to cheaper models while designing the architecture with more expensive ones), leading to even more substantial savings.\n\nEffective model routing needs to optimize for both cost and developer productivity, and not only cost alone. It’s critical to have feedback signals since routers are still early technologies that will need substantial iteration. Our first step here was to log all coding session traces for later evaluation. We want to consider both cost and developer experience – we don’t want to optimize cost at the expense of productivity.\n\nWith Unity AI Gateway, traces for coding agents can be recorded into Unity Catalog. This is extremely sensitive data, and it needs to stay governed by sophisticated tagging and access policies in most mature enterprises.\n\nWe used both AI models and human review to analyze traces to evaluate changes to the router. When we ran this analysis on our own sessions before routing, we observed that a large share of sessions were spending frontier-model money on work that did not need it, simply because the default model was the most expensive. In practice, the way to validate that the router is useful is to continuously monitor the following metrics:\n\nWe believe this is an area with significant opportunity and plan to continue conducting substantial research here. It’s still early, so we have a lot to learn.\n\nOur first challenge has been unreliable benchmarking data that doesn’t match real user behavior. Benchmark tasks are unusually well-behaved, with each arriving as a self-contained statement of work. While routers perform well on such tasks, real sessions often are not like that at all.\n\nSo we’re researching a few new directions to gather more information and try new techniques:\n\nRouting is usually pitched as a way to spend less. While most of our wins come from paying lower prices for easy work, the same machinery can help us decide when to spend more to get a better outcome. Valuemaxxing cuts both ways: take the cheap model when it suffices, and be confident in spending more when the value justifies it.\n\nCoding tools often incentivize users to consume more and more tokens, but what we really want is to optimize productive *output* per dollar, not tokens. Picking a cheaper, faster model when it suffices doesn’t just save money. It also saves time, and it keeps scarce frontier capacity available for the tasks that genuinely need it.\n\nSmart Routing is now available in Beta through Unity AI Gateway. It automatically routes coding tasks to the right model based on complexity, helping teams achieve frontier-level performance with 30%+ cost savings by selecting the best model for every task. For teams looking to reduce AI coding costs without limiting developer choice or productivity, Smart Routing provides an alternative to manually selecting models or relying on blunt spending caps. And with Omnigent, you can extend intelligent routing across models and coding harnesses.\n\nTo get started, visit our docs pages:\n\nLearn more about Unity AI Gateway by visiting our [website](https://www.databricks.com/product/artificial-intelligence/unity-ai-gateway).\n\nSubscribe to our blog and get the latest posts delivered to your inbox.", "url": "https://wpnews.pro/news/smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower-cost-per", "canonical_source": "https://www.databricks.com/blog/smart-routing-unity-ai-gateway-match-frontier-quality-30-lower-cost-task", "published_at": "2026-08-13 17:52:04+00:00", "updated_at": "2026-08-13 19:26:48.051120+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-products", "ai-infrastructure", "machine-learning"], "entities": ["Databricks", "Unity AI Gateway", "Smart Routing", "Omnigent", "Claude Code", "Codex", "Opus 5"], "alternates": {"html": "https://wpnews.pro/news/smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower-cost-per", "markdown": "https://wpnews.pro/news/smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower-cost-per.md", "text": "https://wpnews.pro/news/smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower-cost-per.txt", "jsonld": "https://wpnews.pro/news/smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower-cost-per.jsonld"}}