1. App-to-LLM (North-South Traffic)
App-to-LLM traffic is the direct path from your application to a model provider, like a chatbot calling OpenAI. It needs centralized authentication, per-application rate limits, logging, and guardrails. Because this path handles the raw data payload, the proxy must govern both the connection and the content—managing infrastructure controls (centralized authentication, provider failover, rate limiting) alongside data guardrails (PII sanitization, semantic caching, and prompt injection defense).
Without a Proxy: API keys leak into client repositories, compliance boundaries are breached as sensitive data leaves the network in plaintext, and identical queries repeatedly burn budget by hitting the model instead of a local cache.
2. Agent-to-Agent (East-West Traffic)
Agent-to-agent (A2A) traffic is communication between autonomous agents — one delegating a task to another across services or organizations. Governance here requires establishing verifiable agent identity, enforcing strict per-agent authorization boundaries, and halting cascading, infinite token-depletion loops.
Without a Proxy: Agents execute actions on behalf of other systems without cryptographic identity verification, leaving the enterprise blind to a rapidly growing class of non-human traffic.
3. Model Context Protocol / MCP (Integration Traffic)
MCP traffic connects autonomous agents directly to the local data sources, file systems, and development tools they invoke to execute tasks. Proxying this specialized protocol requires protocol-aware authentication, granular tool-call authorization, and context-window threshold management to safely bound what a model can see and do.
Without a Proxy: Shadow MCP servers spin up across the organization without central access control, granting autonomous agents unmonitored access to internal infrastructure with zero audit trail when a tool call goes wrong.