# Tetrate Adds Ability to Route Inference Requests via AI Gateway > Source: > Published: 2026-07-17 13:12:23+00:00 Tetrate this week [added a token broker service to its artificial intelligence (AI) gateway](https://www.prnewswire.com/news-releases/tetrate-adds-token-brokering-capability-for-ai-code-gen-cost-management-through-agent-router-enterprise-302826151.html) to make it simpler to route inference requests to the large language model (LLM) that is best suited from a capability and cost perspective to process them. Additionally, Tetrate has added a distributed inference control plane to its Tetrate Agent Router Enterprise gateway that enables IT teams to define specific usage policies that redirect a request to an approved alternative rather than returning an error or forcing anyone to rewrite code. David Wang, head of product for Tetrate, said these extensions to the company’s Agent Router Enterprise gateway enable organizations to apply rules to inference requests that could, for example, route many of them to a lower-cost, open source AI model rather than a more expensive proprietary alternative. Additionally, organizations might decide that certain classes of inference requests uniquely require the capabilities of the latest AI models made available by OpenAI or Anthropic, he added. The token brokering capability that Tetrate has added at no additional cost can be invoked via a command line interface (CLI), software development kit (SDK) or application programming interface (API). Based on the open source Envoy AI Gateway project that Tetrate is advancing in collaboration with Bloomberg and Nutanix, the [latest 1.0 version of that open source software has added a single OpenAI-compatible interface](https://www.prnewswire.com/news-releases/envoy-ai-gateway-reaches-v1-0--establishing-the-open-source-standard-for-enterprise-ai-traffic-302808088.html) for invoking AI models along with a native Model Context Protocol (MCP) server and support for token-aware traffic management, centralized credential management, distributed tracing capabilities based on OpenTelemetry GenAI semantic conventions and stable API. Going forward, the project’s maintainers plan to add dollar-based controls alongside existing token-based controls, as well as deeper MCP authorization and identity management capabilities. The Envoy AI Gateway itself is based on the open source Envoy proxy, originally developed by Lyft, that is now being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). In general, the cost of a token continues to decline, but as usage increases, organizations are now moving to control costs. Instead of trying to encourage adoption of AI, also known as tokenmaxxing, organizations are now trying to optimize consumption of AI models. The overall goal is to not reduce the number of use cases for AI, but rather maximize the return on investment, said Wang. Routing inference requests efficiently is especially critical when deploying AI agents, which in the absence of any governance policies will as part of an effort to complete a task without any regard for cost will launch queries against multiple AI models. Many organizations are also now trying to ensure that the consumption of AI models also aligns with any digital sovereignty requirements that might limit what AI models can actually be used. While no two organizations have the same level of AI maturity, it’s now only a matter of time before inference requests are routinely routed to the AI model that an IT team has determined is best capable of handling that request. The challenge, as always, is providing that capability in a way that is seamless to the end user that originally created the request.