InfoWorld reports that AI bill shock is spreading across the industry, driven by opaque large language model (LLM) costs and poor attribution of spend to business value. The article recommends five to…
NVIDIA has released SwitchYard, an open-source routing library that automatically directs AI agent tasks to the most appropriate local or frontier model, claiming it can deliver 50% faster responses a…
NVIDIA's open-source SwitchYard router, built on RouteLLM, can be configured with a locally hosted Nemotron 3.5 Lightning model to cut API costs without losing task quality. On DGX Spark hardware, Nem…
Tracer, a San Francisco AI lab founded by Adam Rida, launched Echo on July 31st, an adaptive inference system that coordinates multiple open-weight models and returns a single answer through an OpenAI…
Manifest, an open-source development platform, shipped an LLM router in March 2024, saw 7,000 cloud users use it for four months, and deprecated it in June, with full shutdown on September 1. The comp…
LLM routers have evolved from a niche infrastructure trick into a mainstream product category, with gateways like OpenRouter, LiteLLM, and Portkey offering API unification and failover, while smart ro…
Microsoft's Azure Kubernetes Service engineering team published a reference implementation on June 29 that separates agent-request routing into three layers: semantic model selection via RouteLLM, gat…
Microsoft released a reference architecture for routing agent traffic on Azure Kubernetes Service that combines the Kubernetes Gateway API Inference Extension, agentgateway, and RouteLLM into one Open…
Spectro Cloud released a TCO calculator for its PaletteAI Inference Launchpad, a turnkey appliance that runs open models on local GPUs with intelligent routing, claiming it can cut inference costs by …
Lynkr, an open-source LLM router, scored 67.65 on the RouterArena benchmark, landing 15th of 27 routers. The router achieved a cost of $0.29 per 1K queries, outperforming GPT-5 as a router at 34× lowe…
Microsoft Copilot now runs on Anthropic's Claude in some features, but users still perceive Claude as smarter because the model is not the product, according to AI researcher Louiza Boujida. Boujida e…
A new benchmark and tutorial series from developer dotnetspark reveals that model-routing systems optimize the wrong axis by routing queries to models without grounding, finding that a cheap local mod…
A new arXiv preprint (2607.13048v1) formalizes when to invoke large language models (LLMs) in streaming inference systems as a risk-based sequential stopping problem, proving six theoretical results i…
Avriz, a cloud platform for coding agents, has built a contextual bandit system that learns to route LLM calls to the cheapest capable model without risking user experience. The system trains exclusiv…
Model routing, which directs simple tasks to cheap models and complex ones to frontier models, can cut AI costs by 40-70% in 2026 while mitigating vendor dependency risks highlighted by Fable 5's susp…
Developer Lore released Wayfinder, an open-source LLM router that determines whether to send a prompt to a local or cloud model by analyzing structural features like length, headings, and code, withou…
Maslul, a new open-source Python library, provides smart LLM routing and provider normalization across Anthropic, Gemini, Grok, and OpenAI, allowing developers to route each request to the right model…