Stop Sending Every SQL Query to Your Most Expensive Model — Build a Router Instead

Victor Horlenko, Head of AI Innovations at Devart, outlined a tiered routing architecture that sends simple SQL queries to cheap models and reserves frontier models for complex reasoning tasks, citing benchmarked costs of about $0.001 per Tier 1 query versus roughly $0.03 for a frontier model. While building schema-aware capabilities into Devart's dbForge AI Assistant, the team found classification accuracy depended heavily on schema context, since queries with ambiguous table names or implicit relationships were misclassified as simple. Well-tuned routing reportedly cuts inference costs 40–60% while keeping escalation under 5%.

If you've built or integrated an AI SQL assistant into your stack, you've likely hit this wall: it works great in the demo, it works great in week one, and then usage scales and the model API bill scales right alongside it. The default fix — swap in a cheaper model everywhere — usually just trades your cost problem for a quality problem. The better fix is architectural: route each query to the model tier its actual complexity requires. SQL queries vary wildly in the reasoning they require. SELECT FROM users WHERE id = 4471 and a cross-schema retention analysis using window functions are both "SQL," but they're not remotely equivalent workloads. Routing both through a frontier model is expensive overkill for the first one. A practical tiering scheme: | Tier | Description | Examples | Model needed | |---|---|---|---| | 1 — Routine | Simple, well-defined | SELECTs, lookups, basic CRUD, syntax fixes | Fast, low-cost model | | 2 — Moderate | Multi-step reasoning | Joins, subqueries, aggregations, optimization hints | Mid-tier model | | 3 — Complex | Deep schema reasoning | Cross-DB queries, window functions, execution-plan tuning, schema refactoring | Frontier model | Benchmarked figures put Tier 1 around $0.001/query versus roughly $0.03/query for a frontier model — a gap that scales linearly with volume. Tier 3 queries also need injected context table relationships, foreign keys, indexes, dialect-specific syntax , which is expensive to carry through every request regardless of tier. This is the stage that determines whether the whole system works. Three implementation options: Rule-based regex/AST : Detect structural signals — table count, join depth, presence of window functions or subqueries. Fast, deterministic, zero model overhead. Handles the obvious cases well. Lightweight classifier model : A small model trained specifically to estimate SQL complexity. Costs a fraction of a cent per call, which easily justifies itself by avoiding unnecessary frontier-model invocations. Can often run locally. Also useful for classifying natural-language prompts before SQL generation even happens. Hybrid : Rules catch the clear cases for free; the classifier handles the ambiguous middle where structure alone doesn't tell you enough. This is the practical sweet spot for most teams. Beyond tier, routing decisions should also account for: Post-execution checks confirm syntax correctness, sane result shapes, and schema consistency. Failures trigger escalation and a rerun at a higher tier. While building schema-aware capabilities into Devart's dbForge AI Assistant https://www.devart.com/dbforge/ai-assistant/ , the team found that classification accuracy depended heavily on schema context — not just query structure. Queries with ambiguous table names or implicit relationships were reliably misclassified as simple and sent to models that couldn't actually resolve them correctly. The fix: feed the classifier schema metadata alongside the query itself, not just the syntax tree. Don't just track average cost — it hides problems. Watch for the escalation tax: a misrouted query means a classifier call + initial model call + failed validation + reroute + second model call. Stack enough of those and you can end up paying more than if you'd just routed to the frontier model directly. Track escalation rate alongside cost per call, not in isolation. Well-tuned routing reportedly delivers 40–60% inference cost reduction while keeping escalation under 5% and preserving quality on complex queries. Pushing past that generally requires self-hosting smaller models for Tier 1 traffic — workable, but it adds real operational overhead infra, monitoring, model lifecycle that not every team needs to take on. This piece draws on an original analysis published on Unite.AI https://www.unite.ai/ai-sql-query-routing-cost-optimization/ by Victor Horlenko, Head of AI Innovations at Devart.