Who controls how your agents use models?

A developer deployed a private model gateway on AWS over two weeks to test whether a centralized layer between agents and models can govern access, spending, data policy, reliability and visibility across ten agents and four teams. The writeup argues that per-agent controls duplicate into ten versions of the same policy, while a gateway centralizes model access checks, usage attribution, budgets, rate limits, input/output screening and retry/fallback policy — but only governs traffic that actually passes through it, and does not constrain what an agent does between model calls.

Post 1 of 8 in the GenAI gateway series. The first agent is easy. You pick a model, give the agent credentials, connect its tools, and test the result. You know which model it calls and where to look when something goes wrong. Then another team builds an agent on a different model. A third team joins. Each agent works, but a month later someone forwards the model bill and asks a short question nobody can answer: which of these agents spent this? Other questions follow. Which agents can use the most expensive model? What data can they send? Who checks that the organisation's rules apply to every team? Customers ask me some version of this all the time. Before this series, I deployed a governed agentic platform from an empty AWS account, and one piece of it kept coming up in every conversation: the layer that every model call passes through. So I spent the last two weeks deploying a gateway privately on AWS, testing each feature, and keeping notes on what broke. This series is those notes. It starts with questions, because they explain why a gateway earns a place in an agent platform. Try answering the ones below for your own setup. Picture ten agents across four teams: a support agent, an incident investigator, a code reviewer, a release checker and an inventory analyst. Between them they call five models, chosen for different tasks and costs. Each agent talks to its model directly. Each team wrote its own client code, picked its own retry settings, and decided for itself what counts as a safe prompt. One task is rarely one model call. An incident investigator asks a model what to check, calls a tool, sends the result back, and decides whether to dig further. A retry or a loop adds more calls. Whatever rules you have must cover the whole sequence. Every team's choices make sense on their own. Look across all ten agents and new jobs appear that no single team owns: access, spending, data policy, reliability and visibility. I hit this early in my own lab. I had one test team in the gateway. When I added a second team for a workload agent, I had to decide its budget, its rate limit and the two models it could call. The gateway gave me one place to configure those controls for both teams. Access defined in ten places is understood in none. A budget you only read at month end is a receipt, not a limit. If you can't verify that a check ran, you can't rely on it. A fallback that returns an answer your agent can't use isn't a fallback. Model logs that don't line up with the agent's own trace give you pieces of a task, not the task. A shared model name turns a code change into a config change. It doesn't remove the need to test each agent on the new model. A shared view makes it easier to spot repeated requests across agents. And a cached answer is only safe if it's right for the agent asking and the data hasn't changed. Every question here is a decision that spans teams: who can use a model, what they can send, how much they can spend, and what happens when a call fails. Leave each decision inside each agent and you maintain ten versions of the same control. That is why agents need a gateway: one service between them and every model they call. Configure it, and the gateway becomes the one place where model access is checked, usage is attributed, budgets and rate limits apply, inputs and outputs are screened, and retries and fallbacks follow one policy. Agents call one interface, and every call is recorded the same way. Teams still own their agents' reasoning, tools and results. The platform owner owns the controls on the model call. A gateway only governs traffic that goes through it. Until agents lose direct access to models, it's a suggestion, not a control. It also doesn't decide whether an agent may run a tool, change a record or act on an answer. A budget on model calls doesn't limit what the agent does between them. Screening a prompt doesn't guarantee the agent resists every injection attempt. And it's a service to run. Someone has to secure it, monitor it and plan for it failing, because every agent depends on it. The next post covers that work. Your agents differ in almost everything except the model call. That's where the shared rules belong. Next in the series: deploying the gateway on AWS.