Dan Temkin
Senior Technical Product Marketing Manager, Kong
Every request returned 200 OK, the task still failed. This is the problem in one sentence. A single user intent now fans out into multiple model calls, tool invocations, MCP (Model Context Protocol) interactions, policy executions, retries, and agent handoffs. Each call can look healthy on its own while the task as a whole hallucinates, times out, or quietly drains your budget. Request-level monitoring can't tell you which, because the answer isn't in any single request.
AI observability is the practice of tracing a full AI task, from first prompt to final answer, across every model, tool, and agent involved, and measuring what it cost and how long it took.
API observability was built for the request #
A call comes in, a response goes out, and you measure status, latency, and error rate. That works because "did the service do its job" maps cleanly onto one exchange.
AI goes beyond the simplistic and requires a deeper perspective.
Where API observability tells you whether your infrastructure responded. AI observability tells you whether the task worked, what it cost, and which model, agent, or tool made the difference. Same discipline, different unit of measurement.
What changes when the unit changes #
The session is the new unit of observability. A session is the full journey behind one outcome. A turn is one exchange within it. See the session and you can see why a task was slow, wrong, or expensive.
Platform teams are the right owners for this. You run the path every AI call takes, so that path is where you see what those calls add up to.
What to measure in AI observability #
Agent observability and LLM (large language model) observability share a core set of signals. With signals that need to be aggregated out of headers, payloads, security tokens and then correlated across the session, not just captured per call.
-
Tokens. Input and output, by model, agent, and consumer.
-
Cost. Spend by team, application, environment, and business unit.
-
Latency. Per turn, and accumulated across the whole session.
-
Errors and retries. Where a task looped, failed, or fell back.
-
Tool and MCP activity. Which tools ran, how often, and for how long.
-
Cache status. What came from cache and what hit the model.
-
Inference Inputs and outputs. What went into each turn and what came back.
What Kong AI Observability does #
Kong AI Observability traces the full journey through models, tools, agents, and policies, directly in the connectivity path. No assembling telemetry across separate systems and providers. Watch the overview, then read on for the highlights.
This content contains a video which can not be displayed in Agent mode
Start with the whole estate
The Overview shows your entire AI estate. See top consumers, performance, and cost, and break down traffic and spend by model, agent, MCP tool, team, or environment. LLM usage, agentic activity, and MCP interactions show up together.
Build the dashboard you need
Prebuilt dashboards give you instant visibility into key metrics, consumer behavior, and performance. When you need something specific, add custom widgets and organize by total spend, request count, or input and output tokens. Group and filter by user, team, environment, application, or business unit. Runaway bills and latency spikes show up before they become incidents.
Share the right view with the right people
Share dashboards and insights with specific people or teams. Role-based access control (RBAC) decides who sees what, so cost and performance data stays protected while developers still get what they need to tune their applications.
Answer one-off questions with Explorer
Answer one-off questions with Explorer
Sometimes you don't need a dashboard. Explorer lets you spin up a single visualization and slice telemetry by dimensions like application, behavior, or cache status. Export the result to share, or add it to an existing dashboard when it's worth keeping.
Go deep with connectivity logs
Search and filter connectivity logs, and toggle between a condensed summary for scanning and the raw data view for troubleshooting. Every call that runs through Kong lands in one place. Nobody reconstructs a conversation by hand.
Trace the session, turn by turn (Coming Soon)
Trace the session, turn by turn (Coming Soon)
This is where Kong AI Observability goes furthest. A single conversation can touch multiple models, tools, and agents before it produces an answer. Kong traces it as one session, so you see the whole path. Drop into the turn where something went wrong and inspect the inputs, outputs, token consumption, cost, and latency, plus the sequence of interactions that produced the result, from policy checks to tool calls.
How is AI observability different from API observability?
API observability measures individual requests. AI observability measures the session, the full journey behind one outcome. A request can succeed while the session fails, so you need both views.
What is session-level tracing?
Session-level tracing follows one conversation or task across every model call, tool call, policy, retry, and agent handoff. You can drill from your whole AI estate into one session, then into the exact turn where something went wrong.
When an AI application fails, knowing that an individual model call returned a 200 isn’t enough. You need to know what led to it. Kong stitches the interactions associated with an AI session together, giving developers a view across the sequence of
Agent spend grows quietly. Usage shows up as an item line on a shared cloud or model invoice, long after the agents that caused it have run, and most teams cannot tell which team, harness, or app drove it. AI Cost Management solves this with brand
Alex Drag
Agentic AI governance is the structured management of the authority an organization hands to autonomous AI systems that plan and take action on its behalf. It covers what an AI agent is allowed to do, which systems it can touch, and how those permis
Kong
Introducing Kong AI Registry: Governed discovery for enterprise AI models, MCP servers, and agents. Most AI agents today operate with a predefined set of connections. Developers configure which models, MCP servers, tools, and other agents they can
Alex Drag
The explosion of AI-native applications is upon us. With each new week, massive innovations are being made in how AI-centric applications are being built. There are a variety of tools developers need to consider, be it supplying live contextual data
Greg Peranich
Most AI cost management starts with infrastructure. A provider can tell you that you consumed a certain number of input and output tokens on a particular model. An observability platform can show requests, latency, tokens, and traces. A cloud cost p
Alex Drag
Turning an API into an MCP server sounds straightforward: expose each API operation as an MCP tool and let the agent decide which one to use. For small APIs, that can work well. But enterprise APIs can contain hundreds of operations. And as agents c
Alex Drag
When an AI application fails, knowing that an individual model call returned a 200 isn’t enough. You need to know what led to it. Kong stitches the interactions associated with an AI session together, giving developers a view across the sequence of
Agent spend grows quietly. Usage shows up as an item line on a shared cloud or model invoice, long after the agents that caused it have run, and most teams cannot tell which team, harness, or app drove it. AI Cost Management solves this with brand
Alex Drag
Agentic AI governance is the structured management of the authority an organization hands to autonomous AI systems that plan and take action on its behalf. It covers what an AI agent is allowed to do, which systems it can touch, and how those permis
Kong
Introducing Kong AI Registry: Governed discovery for enterprise AI models, MCP servers, and agents. Most AI agents today operate with a predefined set of connections. Developers configure which models, MCP servers, tools, and other agents they can
Alex Drag
The explosion of AI-native applications is upon us. With each new week, massive innovations are being made in how AI-centric applications are being built. There are a variety of tools developers need to consider, be it supplying live contextual data
Greg Peranich
Most AI cost management starts with infrastructure. A provider can tell you that you consumed a certain number of input and output tokens on a particular model. An observability platform can show requests, latency, tokens, and traces. A cloud cost p
Alex Drag
Turning an API into an MCP server sounds straightforward: expose each API operation as an MCP tool and let the agent decide which one to use. For small APIs, that can work well. But enterprise APIs can contain hundreds of operations. And as agents c
Alex Drag
When an AI application fails, knowing that an individual model call returned a 200 isn’t enough. You need to know what led to it. Kong stitches the interactions associated with an AI session together, giving developers a view across the sequence of
Agent spend grows quietly. Usage shows up as an item line on a shared cloud or model invoice, long after the agents that caused it have run, and most teams cannot tell which team, harness, or app drove it. AI Cost Management solves this with brand
Alex Drag
Agentic AI governance is the structured management of the authority an organization hands to autonomous AI systems that plan and take action on its behalf. It covers what an AI agent is allowed to do, which systems it can touch, and how those permis
Kong
Introducing Kong AI Registry: Governed discovery for enterprise AI models, MCP servers, and agents. Most AI agents today operate with a predefined set of connections. Developers configure which models, MCP servers, tools, and other agents they can
Alex Drag
The explosion of AI-native applications is upon us. With each new week, massive innovations are being made in how AI-centric applications are being built. There are a variety of tools developers need to consider, be it supplying live contextual data
Greg Peranich
Most AI cost management starts with infrastructure. A provider can tell you that you consumed a certain number of input and output tokens on a particular model. An observability platform can show requests, latency, tokens, and traces. A cloud cost p
Alex Drag
Turning an API into an MCP server sounds straightforward: expose each API operation as an MCP tool and let the agent decide which one to use. For small APIs, that can work well. But enterprise APIs can contain hundreds of operations. And as agents c
Alex Drag
Ready to see Kong in action? #
Get a personalized walkthrough of Kong's platform tailored to your architecture, use cases, and scale requirements.