AI agents are changing the observability landscape, introducing new concerns about response quality and AI spend. An agent can complete a request without an error and still retrieve the wrong information or produce a low-quality response. Multistep workflows may include several prompts, tool calls, retrievals, retries and completions, making it difficult to determine where quality degraded and what drove token consumption and cost. Agents also produce more telemetry than traditional application requests, increasing the cost and complexity of retaining the data needed for investigation. To manage costs, teams often resort to sampling or dropping telemetry, leaving gaps when they need the data most.
Agent Observability (coming soon to private preview) in Observe by Snowflake is designed to tackle these emerging challenges. It enables teams to monitor, debug and improve AI agents and LLM applications by tracing how interactions unfold and connecting agent behavior with cost and quality.
Agent Observability will provide the instrumentation, metrics, tooling and UI teams need to debug agent interactions, isolate performance and quality issues, identify what drives AI spend and improve business outcomes. Built natively on Snowflake, Observe is designed to retain and analyze agent telemetry cost-effectively at scale. Teams can then correlate these operational signals with business data on Snowflake to understand which agents and workflows deliver meaningful outcomes, improve those that fall short and direct AI investment toward areas that create the most value.
What's coming in Agent Observability #
With Agent Observability, teams will be able to:
- Instrument agents using an optional OpenTelemetry-compliant SDK or existing instrumentation: Capture prompts, completions, retrievals, tool calls, token usage and session IDs using an SDK that follows emergingOpenTelemetry (OTel) GenAI semantic conventions. The SDK includes tested and verified support for LangChain, the Anthropic Agents SDK and the OpenAI Agents SDK. Teams can also instrument agents themselves or use custom frameworks, then send properly instrumented traces to Observe through an OTLP endpoint.
- Search and debug conversations in the Agent Explorer: Search across traces, sessions and conversations to inspect how an agent behaved, which tools it called and where a failure or unexpected response started.
- Monitor performance, usage and cost: Analyze metrics derived from agent traces, including latency, errors and cost.
- Run online LLM-as-judge evaluations for production traffic: Surface hallucinations, guardrail failures and poor-quality responses. Connect evaluation results to the underlying spans and traces, and monitor quality over time.
- Track offline evaluation results: Send results from existing development and testing workflows into Observe as traces or metrics for visualization.
- Query telemetry programmatically****: Use APIs to query traces, metrics and online evaluation results from third-party agents and automated workflows.
Built for the scale and economics of agent telemetry #
As organizations deploy more agents and expand their use cases, the telemetry needed to understand those agents expands quickly. This growth is driven by the structure of agent interactions. An agent trace records the sequence of steps behind an agent interaction, including model calls, retrievals, tool invocations, retries and intermediate outputs — which means agent traces run longer, contain larger spans and generate more spans than traditional application traces.
This creates a data challenge for many teams. They need enough detail to reconstruct important interactions, but retaining that much telemetry can substantially increase observability costs. Some teams are already concerned that growing volumes will force them to sample or discard data, leaving gaps in investigation.
Observe is built for high-volume, complex event data, giving Agent Observability the architecture needed to retain and query high-fidelity agent telemetry.
With Observe, teams are able to:
- Keep long-running traces and conversations complete : Observe extends the event-based approach used in itsApplication Performance Monitoring product to agent interactions, storing spans as events and querying them by trace ID and conversation ID so late-arriving spans and long gaps remain part of the complete interaction.
- Query large, context-rich spans : Snowflake's native support for complex, semi-structured event data makes Observe well-suited to storing and querying prompts, model responses, retrieved context, and tool inputs and outputs without stripping away valuable detail.
- Retain growing telemetry volumes cost-effectively : Cloud object storage is designed to provide a lower-cost way to store detailed agent telemetry, while Observe's Telemetry Lakehouse Foundation separates storage from compute so teams can retain more data and scale compute independently for search, analysis and debugging.
Debug and investigate complete agent interactions #
When an agent produces a poor result, the final response rarely tells the complete story. Engineers need to understand the context the agent received, the sequence of model calls, retrievals, tool invocations and retries it executed, the intermediate outputs produced at each step and where the workflow began to diverge from the intended outcome.
Customers tell us that this information is often fragmented across the application, agent framework, model provider and supporting services. Some teams lose visibility once a request leaves their environment, turning user complaints about latency, errors or unexpected behavior into investigations that can take hours or days. They need a way to trace and debug the end-to-end agent workflow in one place.
Observe is designed to capture the context behind these interactions and organizes it into agent-aware views across conversations, sessions, traces and individual spans. With the Agent Explorer, teams can search across agent interactions using captured attributes, then move from a conversation into its underlying traces. They can inspect prompts, completions, retrieved context, tool activity and retries to identify where a failure or unexpected response began.
A delayed or incorrect response may originate within the agent workflow, such as during model processing, retrieval, tool use or orchestration. It may also result from an issue in a supporting application, service or infrastructure component. Observe's Context Graph connects agent telemetry with related applications, services, APIs, databases and infrastructure, helping teams correlate agent spans with upstream and downstream dependencies rather than treating the agent workflow in isolation.
Evaluation quality as part of production observability #
Traditional observability can show that a request completed quickly and without errors, but it cannot determine whether the response was accurate, relevant or useful. For nondeterministic systems, quality needs to become a measurable production signal.
Observe supports online LLM-as-judge evaluations against production traces, turning results into metrics teams can monitor over time. Because each result links to the underlying interaction, engineers can trace quality issues to the prompt, retrieved context, tool selection, supporting services or model response and assess whether changes improve production behavior.
Teams can also send results from existing offline evaluation workflows into Observe as traces or metrics, providing a shared view of development results and production behavior without replacing their preferred tools.
Connect agent performance and AI spend to quality and business outcomes #
In conversations with customers, we hear that token totals alone are not enough to determine whether AI spend is producing value. Leaders want to attribute consumption to an application or project and understand what that spend accomplished. This requires visibility into the factors that affect the cost and performance of an agent interaction, including model selection, prompt size, retrieved context, tool use, retries and workflow design. Engineering teams can analyze this data to identify inefficiencies such as unnecessary retries or expensive model choices that increase costs without improving quality. For example, a poorly configured evaluation workflow can consume available AI budget much faster than expected.
Observe derives latency, error, token usage and estimated cost signals from agent traces, helping teams compare agents, models and workflows and identify inefficient execution paths. A lower-cost model may not be the better choice if it produces more failed evaluations, while a higher-cost model may not justify its expense if it delivers no meaningful quality improvement.
This is where the combination of Observe and Snowflake capabilities shines. Observe helps teams make better tradeoffs between performance, quality and cost when it comes to optimizing AI spend; Snowflake adds the business context needed to understand the impact of those decisions. Teams can correlate agent behavior with customer experience, resolution rates, conversion and revenue for customer-facing agents, or with task completion and employee productivity for internal applications. This helps organizations improve agents based on the outcomes they produce and direct AI investment toward what creates value.
Preserve flexibility with open standards #
Agent architectures are evolving quickly, and observability should not restrict which models, frameworks or gateways teams use. Customers tell us they are already working across custom frameworks, specialist evaluation products and cloud-native services. They need a way to gain consistent visibility across these environments without being locked into another proprietary data format or rebuilding their instrumentation as their architecture changes.
Observe supports OTel-native collection through an OTLP endpoint, while its optional OTel-compliant SDK provides a convenient way to capture and correlate agent telemetry using emerging GenAI semantic conventions. This commitment to open standards extends to storage through Apache Iceberg™. The Observe experience can operate on supported telemetry stored in Iceberg tables, while compatible engines can access the same data through an open REST catalog.
Together, OTel and Iceberg help teams preserve flexibility across instrumentation, storage and query engines while retaining the investigation and monitoring capabilities of Observe.
Get started today #
Agent Observability in Observe is coming soon to Private Preview. If your team needs to investigate agent interactions, evaluate production quality, optimize AI spend or retain agent telemetry at scale, we want to hear from you.
- Register for our Agent Observability launch event on Oct. 22, featuring insights from Jeremy Burton, General Manager of Observability at Snowflake and Christian Kleinerman, Executive Vice President of Product at Snowflake
- Reach out to your Observe account team to request private preview access
This article contains forward-looking statements, including about our future product offerings, and is not a commitment to deliver any product offering. Actual results and offerings may differ and are subject to known and unknown risks and uncertainties. See our latest 10-Q for more information.
Apache®, Apache Iceberg™ and related marks are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.