Trail – signed OpenTelemetry spans for AI agents Trail, a new Python SDK, captures AI agent activity as signed OpenTelemetry spans, adding agent-aware tool taxonomy, MCP injection flagging, and skill hashing to detect silent substitution, while exporting via OTLP to any backend without storing data itself. The SDK signs each session with Ed25519 and supports backends like Grafana, Honeycomb, Datadog, and Chronosphere, aiming to answer questions about tool execution order, prompt injection attempts, and skill code changes in production. Signed OpenTelemetry GenAI spans for AI agents. Capture, normalize, verify — bring your own backend. Trail is a Python SDK that captures what AI agents actually do — every LLM call, tool invocation, MCP call, and skill execution — as OpenTelemetry spans with a small Trail extension namespace. It signs each session with Ed25519 and exports via OTLP to any OTel backend Grafana, Honeycomb, Datadog, Chronosphere, ... . Trail does not store, query, or dashboard. Storage and query are your existing backend's job. When an agent misbehaves in production, three questions are surprisingly hard to answer: Which tool ran, in what order, with what inputs? Existing tracers are LLM-call-shaped, not agent-shaped. Was that MCP server response trying to inject instructions? No mainstream tracer flags this. Is this skill the same code it was yesterday? Skill substitution leaves no trace by default. Trail adds the three things that are missing: an agent-aware tool taxonomy internal / mcp / skill / builtin , MCP injection flagging on tool responses, and a skill hash that detects silent substitution — all as standard OpenTelemetry spans, so any OTel backend ingests them with no translation layer. Trail models an agent run as an OpenTelemetry span tree — one invoke agent root span per session, with every LLM call, tool, MCP call, and skill nested underneath — and layers a trail. attribute namespace on top. That structure, plus three purpose-built attributes, is what turns each question above into a query. Which tool ran, in what order, with what inputs? Every tool invocation becomes an execute tool span tagged with gen ai.tool.name and trail.tool type internal / mcp / skill / builtin — the agent-shaped distinction a plain LLM tracer never draws. Order and nesting come from the OpenTelemetry SDK's contextvars propagation, which stays correct across async / await and concurrent asyncio tasks, so each span attaches to the right parent. Inputs and outputs are recorded as trail.input hash / trail.output hash SHA-256, computed off the hot path plus a sensitivity flag — tamper-evident identity of the payloads without storing the payloads themselves. gen ai.operation.name = "execute tool" gen ai.tool.name = "get customer record" trail.tool type = "mcp" trail.input hash = "sha256:..." Was that MCP server response trying to inject instructions? When Trail wraps an MCP call tool , it runs the response through a YAML injection ruleset — instruction-override, system-prompt injection, role override, credential-exfil phrasing override via TRAIL MCP RULES — and stamps the span with trail.mcp.injection flag . A response that says "ignore your previous instructions and…" lands as an ordinary span with trail.mcp.injection flag = true , next to trail.mcp.server id for provenance. Is this skill the same code it was yesterday? wrap skill records trail.skill.hash — a SHA-256 over the skill's source trail.skill.hash method = "source" , with a qualname-fallback for C-extensions and lambdas . Same skill → same hash; a silent swap → a different hash on today's span versus yesterday's. Diff the attribute across two sessions and substitution is visible. Then you ask where you already look. Trail only captures — the questions get answered in your backend. In dev mode that's the session JSONL ~/.trail/sessions/{trace id}.jsonl , and trail verify-export proves none of it was altered after the fact and pinpoints the span if it was . In prod, it's an ordinary attribute filter — trail.tool type = "mcp" AND trail.mcp.injection flag = true — in Grafana, Honeycomb, or Datadog. pip install 'trail-otel openai ' python import openai import trail trail.auto instrument detects openai, instruments it with trail.session agent id="content-pipeline" : client = openai.OpenAI client.chat.completions.create model="gpt-4o", messages= ... That's it. By default Trail writes spans to ~/.trail/sessions/{trace id}.jsonl and a short summary to stderr. Zero infrastructure. To ship to your OTel backend instead: export TRAIL EXPORT=otlp export OTEL EXPORTER OTLP ENDPOINT=https://your-collector:4317 Async OpenAI AsyncOpenAI is instrumented automatically by the same trail.auto instrument call. Trail ships a trail-hook console script. Wire it into ~/.claude/settings.json . A single binary handles all three events — it reads the event name from Claude Code's stdin payload and dispatches internally. { "hooks": { "PreToolUse": { "matcher": " ", "hooks": { "type": "command", "command": "trail-hook" } } , "PostToolUse": { "matcher": " ", "hooks": { "type": "command", "command": "trail-hook" } } , "SessionEnd": { "hooks": { "type": "command", "command": "trail-hook" } } } } Already have hooks? Claude Code's hooks.