Unify Claude observability into Databricks with native OpenTelemetry ingestion. Use collected traces with MLflow to find recurring failures and improve the agents.
An agent can return a plausible answer while missing an important finding or reading the same source repeatedly. The answer alone won't explain why. Traces show the model and tool calls behind it, so you can investigate the failure, find related cases, and test whether a change helps.
Databricks MLflow is now a supported destination for native OpenTelemetry export from the Claude Platform.. With native export to Databricks MLflow, developers using the Anthropic SDKs, the Messages API and Managed Agents can send native traces to Databricks and use MLflow to debug their agents and evaluate changes.
MLflow Tracing is built on OpenTelemetry, an open standard for collecting and exporting service signals. A trace records an agent’s execution as individual steps, called spans, such as model requests and tool calls. Claude’s native instrumentation generates these spans and exports them using the OpenTelemetry Protocol.
Databricks receives the spans through Zerobus Ingest and stores them in Unity Catalog tables linked to an MLflow experiment. You can inspect Claude traces alongside traces from other instrumented services, with the same tools for debugging and evaluation. Unity Catalog controls access to the data, which you can also query with SQL. With the combination of Zerobus and Unity Catalog, enterprises can get two crucial benefits: low-latency trace ingestion at high QPS plus the ability to govern access to the sensitive data stored in agentic traces.
In Databricks, open the Experiments tab and click Create Experiment. Under Trace storage location, keep Unity Catalog selected. Choose an existing catalog and schema. This creates new MLflow experiments linked to the Unity Catalog table that stores OpenTelemetry traces.
The detailed setup guide covers permissions and SQL warehouse requirements.
Set the Databricks destination using your workspace ID, region, and spans table. The Zerobus connection guide explains how to obtain a table-scoped OAuth token.
The new native OTel integrations cover Anthropic SDKs, the Messages API, and Managed Agents. For the Anthropic SDK, install the packages below and configure an OTel tracer provider and exporter to create spans for Anthropic SDK calls. Follow Claude Platform OpenTelemetry guide to send traces from the Messages API and Claude Managed Agent your Databricks destination.
Claude Code and Claude Agent SDK also support OpenTelemetry. To send their traces to the same destination, set these additional variables before starting your application:
Run your Claude agent, then open Traces in the experiment. Open a trace to inspect its model calls, tool activity, timing, and captured inputs and outputs. Note that inputs and outputs are captured only if you opt in by setting content mode to “full”.
Once Claude traces arrive in MLflow, you can use them throughout development. A trace provides a record for debugging, a case for expert review, and evidence for evaluating the next version of an agent.
An unexpected answer can come from missing context, a failed tool call, or how the model used a tool result. MLflow’s Analyze trace action brings Genie Code into the investigation. It examines the recorded execution and helps explain where things went wrong.
Ask follow-up questions about missing context, failed tool calls, or expensive steps, to inspect the relevant context behind the explanation. This helps you decide whether to revise instructions, improve a tool, or change the context available to the agent.
MLflow Issues, available in Beta, uses Genie Code to identify recurring quality and operational problems. It examines responses, tool calls, timing, errors, and assessments, then groups related findings with links to the supporting traces. Examples include repeated searches that waste tokens, missing arguments that cause tool failures, and responses that don’t follow the required format.
You can schedule detection to keep this analysis running as new traces arrive. Teams can choose how often it runs, use metadata or tags to focus on particular traffic, and customize the types of problems it looks for. After a fix, an issue monitor can track whether the problem returns. MLflow creates a scorer for the issue, evaluates recent traces to establish a baseline, and scores a sample of new traces to measure its frequency over time.
Some quality judgments require domain knowledge. MLflow Review Queues bring that expertise into the development process through selected traces and questions tailored to the agent.
Reviewers can assess the recorded request and response, explain what is missing, and provide an expected answer. Their feedback helps developers refine the agent and establish reference answers for subsequent evaluations.
A change to instructions, tools, or skills can fix one failure while introducing another. Traces also make agent behavior measurable. Run MLflow Agent Evaluation on Claude traces to test whether a change to prompts, skills, tools, improve the agent or regress.
For example, the following code executes an LLM judge to examine whether an agent makes redundant tool calls. When the original and revised agents have run against the same cases, their evaluation runs can be compared using the same judges. Click Compare in MLflow UI to see scores and responses side by side, with the underlying traces available to explain remaining failures.
The same approach works for checks such as whether an agent followed a required workflow or used the appropriate tools. Testing and Refining Claude Code Skills with MLflow demonstrates how these checks and their rationales can guide improvements to skills.
The following resources cover connecting Claude to Databricks MLflow and using its traces during development:
The first CTA item still needs the dedicated Claude integration-guide URL before publication.
Subscribe to our blog and get the latest posts delivered to your inbox.