Show HN: Traccia – Vendor-neutral runtime control for AI agents

Traccia, a vendor-neutral Python SDK for runtime control of AI agents, is now available on PyPI, offering observability, evaluation and policy enforcement built on OpenTelemetry. The SDK's @govern decorator can deny or reshape LLM and tool calls at runtime using policies including Spend Cap, Model Boundary and Loop Cap, and it auto-instruments OpenAI, Anthropic and Gemini (google-genai) plus requests and HTTP libraries, with integrations for LangChain, CrewAI and the OpenAI Agents SDK. Traccia also ships offline evaluation via evaluate(), prompt management with load_prompt and prefetch_prompts, and EU AI Act and HIPAA-oriented governance evidence with PII/PHI redaction helpers.

AI agents can act. Traccia controls what they're allowed to do. Traccia is a production-ready Python SDK for observing, evaluating, and enforcing policies on AI agents and LLM applications at runtime. Built on OpenTelemetry, Traccia provides distributed tracing, token and cost tracking, evaluation, governance evidence, and runtime policy enforcement across AI agents, agentic workflows, and multi-agent systems. Framework-agnostic and built for OpenAI Agents, LangGraph, CrewAI, and LLM applications in production. Traccia https://pypi.org/project/traccia/ is available on PyPI. - Runtime Policies : @govern can deny or reshape LLM or tool calls at runtime using policies such as Spend Cap, Model Boundary, and Loop Cap - Guardrail Detection : Passive detection of AI safety controls, provider-native safeguards, and custom guardrails - Security Controls : No secrets in logs and configurable data truncation - Automatic Instrumentation : Auto-patch OpenAI, Anthropic, Gemini google-genai , requests, and HTTP libraries - Framework Integrations : Support for LangChain, CrewAI, and OpenAI Agents SDK - LLM-Aware Tracing : Track tokens, costs, prompts, completions, and latency automatically - OpenTelemetry Metrics : Emit OTEL-compliant metrics for accurate token and cost tracking independent of sampling - Decorator-Based Tracing : Trace any function with the @observe decorator - Multiple Exporters : OTLP-compatible export to Grafana Tempo, Jaeger, Zipkin, SigNoz, Console, or File - Production-Ready Architecture : Rate limiting, error handling, configuration validation, and reliable flushing - High Performance : Efficient batching, async support, and low-overhead instrumentation - Offline Evaluation : evaluate runs a task and scorers over a dataset and saves an experiment you can open, compare, and attach on promote - Prompt Management : load prompt / prefetch prompts with cache, stale-while-revalidate, fallback, and traccia.prompt. span identity - Zero-Configuration Setup : Simple init call with automatic configuration discovery - Type-Safe Configuration : Full Pydantic validation and configuration management - AI Governance and Compliance : EU AI Act and HIPAA-oriented evidence, transparency records, integrity verification, and PII/PHI redaction helpers pip install traccia python from traccia import init, observe Initialize auto-loads from traccia.toml if present init Trace any function @observe def my function x, y : return x + y That's it Traces are automatically created and exported result = my function 2, 3 python from traccia import init, observe from openai import OpenAI init Auto-patches OpenAI client = OpenAI @observe as type="llm" def generate text prompt: str - str: response = client.chat.completions.create model="gpt-4", messages= {"role": "user", "content": prompt} return response.choices 0 .message.content Automatically tracks: model, tokens, cost, prompt, completion, latency text = generate text "Write a haiku about Python" python from traccia import init from google import genai init Auto-patches google-genai client = genai.Client response = client.interactions.create model="models/gemini-flash-latest", input="Write a haiku about Python", Fetch prompts from your Traccia prompt registry by name and deploy label. Compiling fills {{variables}} and stamps the active span with prompt identity. python from traccia import init, observe, load prompt, prefetch prompts init api key="...", prompt cache ttl s=60 prefetch prompts "support-reply" optional warm-up @observe as type="llm" def reply question: str - str: prompt = load prompt "support-reply", label="production", fallback={ "type": "chat", "messages": {"role": "system", "content": "You are a helpful assistant."} , }, messages = prompt.compile question=question pass messages to your LLM client return messages -1 "content" Requires the Traccia platform https://app.traccia.ai : create a workspace API key under Settings → API Keys . Tracing and prompt fetch use the same key and default to https://api.traccia.ai . Pass an explicit fallback so agents can still run if a fetch fails. See Prompts in the SDK https://traccia.ai/docs/sdk/prompts . evaluate runs a task and scorers over a dataset. By default the run is saved as an experiment you can open in the app under Evaluate → Experiments , compare against other runs, and attach when you promote a prompt. Use the same workspace API key as tracing and load prompt . python from traccia import init, evaluate, load prompt init api key="..." prompt = load prompt "support-reply", label="production" def task inp : messages = prompt.compile inp return call model messages result = evaluate "support-reply-v3", data="support-golden", platform dataset name or UUID task=task, scorers= "exact match" , builtins, platform names/ids, or callables prompt="support-reply", labels the experiment cell with this prompt print result.summary print result.url - Persist is on by default. Set persist=False for a local-only loop no experiment URL . - Inline rows work as {"input": {...}, "expected": "..."} . With persist on, Traccia still saves a full experiment helper datasets named sdk-eval/... are hidden on Evaluate → Datasets unless you turn on Show SDK-Created . - Builtins run in-process: exact match , contains , json valid . Mix in platform scorers LLM-as-judge, Jev Decision, code by name or UUID. Pass provider keys= for judges typesafe for Jev . A Jev score keeps each question on the experiment. - One throwing row does not abort the run. That cell records error ; other items still score. Configuration and API failures raise EvaluateError . - If the first task argument is named row , item , example , or case , Python passes the full row. Any other name including input receives only the input dict. See Evaluate in the SDK https://traccia.ai/docs/sdk/evaluate and the walkthrough Run Experiments From Code https://traccia.ai/docs/guides/sdk-evaluate . traccia eval runs a task script, saves an experiment, and writes summary.json . The process exit code is the gate: 0 pass, 1 quality miss, 2 bad command or baseline, 3 untrustworthy evidence, 4 the experiment did not save, 5 the summary file was not written. With no pass-rate bar, a green check is Execution Only . Add --min-pass-rate or --max-drop for a Quality Gate . --max-drop 0.05 is five percentage points. --max-task-cost-increase 0.2 is 20 percent. The script defines task and, for a function in the file, a scorers dict passed as --scorer local:name . A local scorer cannot pass --max-drop . Pass --prompt with --prompt-version or --prompt-label when the experiment should record a prompt version. Platform LLM-judge and Jev scorers read OPENAI API KEY , ANTHROPIC API KEY , GEMINI API KEY or GOOGLE API KEY , GROQ API KEY , and TYPESAFE API KEY , or a JSON object in TRACCIA PROVIDER KEYS . Those values are not written into the summary. Each Jev question is gated on its own pass rate. Execution Only means no pass-rate gate; cost and latency gates can still apply. traccia eval experiments/support agent.py \ --output summary.json \ --dataset support-golden \ --scorer exact match \ --min-pass-rate 0.9 \ --baseline-prompt support-reply \ --max-drop 0.05 See Run Evals In CI https://traccia.ai/docs/guides/ci-evals . Create a callback handler and pass it to config={"callbacks": traccia handler } . Install the optional extra: pip install traccia langchain . python from traccia import init from traccia.integrations.langchain import CallbackHandler or TracciaCallbackHandler from langchain openai import ChatOpenAI init Create Traccia handler no args traccia handler = CallbackHandler Use with any LangChain runnable llm = ChatOpenAI model="gpt-4o-mini" result = llm.invoke "Tell me a joke", config={"callbacks": traccia handler } Spans for LLM/chat model runs are created automatically with the same attributes as direct OpenAI instrumentation model, prompt, usage, cost . Note: pip install traccia langchain installs traccia plus langchain-core ; you need this extra to use the callback handler. If you already have langchain-core e.g. from langchain or langchain-openai , base pip install traccia may be enough at runtime, but traccia langchain is the supported way to get a compatible dependency. Traccia automatically detects and instruments the OpenAI Agents SDK when installed. No extra code needed: python from traccia import init from agents import Agent, Runner init Automatically enables Agents SDK tracing agent = Agent name="Assistant", instructions="You are a helpful assistant" result = Runner.run sync agent, "Write a haiku about recursion" Configuration : Auto-enabled by default when openai-agents is installed. To disable: init openai agents=False Explicit parameter OR set environment variable: TRACCIA OPENAI AGENTS=false OR in traccia.toml under instrumentation : openai agents = false Compatibility : If you have openai-agents installed but don't use it e.g., using LangChain or pure OpenAI instead , the integration is registered but never invoked—no overhead or extra spans. Traccia automatically instruments CrewAI https://docs.crewai.com/ when it is installed in your environment. python from traccia import init from crewai import Agent, Task, Crew, Process init Auto-enables CrewAI tracing when CrewAI is installed researcher = Agent role="Research Analyst", goal="Research a topic", llm="gpt-4o-mini" task = Task description="Research Shawn Michaels", agent=researcher crew = Crew agents= researcher , tasks= task , process=Process.sequential, verbose=True result = crew.kickoff Traccia will create spans for the crew crewai.crew.kickoff , each task crewai.task. , agents crewai.agent. , and underlying LLM calls, which nest under the existing OpenAI spans. Configuration : Auto-enabled by default when crewai is installed. To disable: init crewai=False Explicit parameter OR set environment variable: TRACCIA CREWAI=false OR in traccia.toml under instrumentation : crewai = false Traccia includes a passive guardrail detection engine that runs as a span processor — no runtime enforcement, no changes required to existing agent code. It inspects every span as it ends, classifies guardrail signals into structured findings, and writes results back onto spans so they appear in any configured exporter. Detection is automatic once traccia.init is called. The processor runs three tiers against every span's attributes: | Tier | Source | Confidence | How | |---|---|---|---| | A — Explicit | explicit | high | @observe as type="guardrail" or guardrail span | | B — Provider-native | provider native | high / medium | LLM finish reason, stop reason, safety ratings | | C — Heuristic | heuristic | always low | Denial keywords in tool error messages | At the end of each trace when the root span ends , the processor evaluates which guardrail categories should exist given the agent's observed capabilities LLM calls, tool use, user-provided text and reports which are missing. Option 1: guardrail span context manager — recommended for inline checks python from traccia.guardrails import guardrail span with guardrail span "pii check", category="pii", enforcement mode="warn" as span: result = run pii check user input span.set attribute "guardrail.triggered", result.found pii Option 2: @observe as type="guardrail" decorator — for function-level guardrails If your guardrail function returns a bool , guardrail.triggered is set automatically: python from traccia import observe @observe as type="guardrail", attributes={ "guardrail.name": "prompt injection check", "guardrail.category": "prompt injection", "guardrail.enforcement mode": "block", } def check injection text: str - bool: return any kw in text.lower for kw in INJECTION KEYWORDS True → triggered blocked , False → not triggered For non-bool returns, set guardrail.triggered manually on the current span. Batch pipelines or internal-only agents often get flagged for prompt injection / input validation because they make LLM calls with llm.prompt . Suppress specific categories for a run: python from traccia.guardrails import guardrail span Convenience: suppress missing on guardrail span with guardrail span "root", category="unknown", suppress missing= "prompt injection", "input validation" : run batch pipeline Or directly on any span from traccia.guardrails import ATTR GUARDRAIL SUPPRESS MISSING span.set attribute ATTR GUARDRAIL SUPPRESS MISSING, "prompt injection" Per span when a guardrail signal is found : - guardrail.finding.count — number of findings on this span - guardrail.findings — JSON array of GuardrailFinding objects On the root span aggregated summary of the entire run : - guardrail.summary — full GuardrailSummary JSON - guardrail.summary.coverage confidence — "high" , "medium" , or "low" - guardrail.summary.missing count — number of expected-but-missing guardrail categories - guardrail.summary.detected categories — list of detected category strings These signals are detected automatically from LLM span attributes — no annotation required: | Provider | Signal | Attribute | |---|---|---| | OpenAI | finish reason = "content filter" | llm.finish reason | | Azure OpenAI | finish reason = "content filtered" | llm.finish reason | | Google GenAI | finish reason = "SAFETY" | llm.finish reason | | Anthropic | stop reason = "content filter" | llm.stop reason | | Anthropic | Policy violation error message | error.message + llm.vendor=anthropic | | Google/LangChain | "blocked": true or "probability": "HIGH" in safety ratings | llm.safety ratings | Tier A/B stay on. Turn off tool-error keyword heuristics if you do not want Tier C at all: init guardrail heuristics=False Or TRACCIA GUARDRAIL HEURISTICS=false , or in traccia.toml : instrumentation → guardrail heuristics = false . - Guardrails running outside the traced process API gateways, proxies, external validators are invisible unless they write span attributes. - A guardrail that exists but never fires cannot be distinguished from a missing guardrail. Only explicit annotation proves presence. - Prompt injection detection requires an explicit span — the model's output alone cannot reliably indicate whether a check ran. - Capability inference handles user text is inferred from llm.prompt being present; batch pipelines and user-facing agents look the same. Use suppression to opt out. Traccia merges configuration from multiple sources with the following priority highest to lowest : 1. Explicit parameters — init endpoint="...", agent id="..." or start tracing ... 2. Environment variables — TRACCIA ENDPOINT , TRACCIA AGENT ID , etc. 3. Config file — traccia.toml current directory or ~/.traccia/config.toml 4. Defaults — Built-in SDK defaults Example : If you set TRACCIA ENDPOINT in your environment and pass endpoint=... to init , the explicit parameter wins. Create a traccia.toml file in your project root: traccia config init This creates a template config file: tracing API key — required for the Traccia platform, not needed for local OTLP backends api key = "" Endpoint URL for OTLP trace ingestion default: Traccia platform For local OTLP backends use e.g. endpoint = "http://localhost:4318/v1/traces" endpoint = "https://api.traccia.ai/v2/traces" sample rate = 1.0 0.0 to 1.0 auto start trace = true Auto-start root trace on init auto trace name = "root" Name for auto-started trace use otlp = true Use OTLP exporter service name = "my-app" Optional service name service role has no env var — pass via init service role="orchestrator" exporters Only enable ONE exporter at a time enable console = false Print traces to console enable file = false Write traces to file file exporter path = "traces.jsonl" reset trace file = false Reset file on initialization instrumentation enable patching = true Auto-patch libraries OpenAI, Anthropic, requests enable token counting = true Count tokens for LLM calls enable costs = true Calculate costs openai agents = true Auto-enable OpenAI Agents SDK integration crewai = true Auto-enable CrewAI integration guardrail heuristics = true Tier C heuristic guardrail detection tool error keywords auto instrument tools = false Auto-instrument tool calls experimental max tool spans = 100 Max tool spans to create max span depth = 10 Max nested span depth rate limiting Optional: limit spans per second max spans per second = 100.0 max queue size = 5000 Max buffered spans max block ms = 100 Max ms to block before dropping max export batch size = 512 Spans per export batch schedule delay millis = 5000 Delay between batches metrics enable metrics = true Enable OpenTelemetry metrics metrics endpoint = "" Defaults to {traces base}/v2/metrics metrics sample rate = 1.0 Metrics sampling rate 1.0 = 100% runtime Optional runtime metadata agent identity: prefer init agent id=..., agent name=..., env=... or TRACCIA env session id = "" user id = "" tenant id = "" project id = "" agent id = "" Single-agent: set in code or TRACCIA AGENT ID agent name = "" env = "" e.g. production, staging, dev logging debug = false Enable debug logging enable span logging = false Enable span-level logging advanced attr truncation limit = 1000 Max attribute value length If you do not set endpoint in config, environment, or when calling init / start tracing , the SDK uses the Traccia platform by default https://api.traccia.ai/v2/traces . You can override it to send traces to your own OTLP-compatible backend. The default is defined in traccia.config : DEFAULT OTLP TRACE ENDPOINT . The alias DEFAULT ENDPOINT is kept for backward compatibility same value . Traccia is fully OTLP-compatible and works with: - Grafana Tempo - http://tempo:4318/v1/traces - Jaeger - http://jaeger:4318/v1/traces - Zipkin - Configure via OTLP endpoint - SigNoz - Self-hosted observability platform - Traccia Platform - https://api.traccia.ai/v2/traces requires API key All config parameters can be set via environment variables with the TRACCIA prefix: Tracing : TRACCIA API KEY , TRACCIA ENDPOINT , TRACCIA SAMPLE RATE , TRACCIA AUTO START TRACE , TRACCIA AUTO TRACE NAME , TRACCIA USE OTLP , TRACCIA SERVICE NAME Exporters : TRACCIA ENABLE CONSOLE , TRACCIA ENABLE FILE , TRACCIA FILE PATH , TRACCIA RESET TRACE FILE Instrumentation : TRACCIA ENABLE PATCHING , TRACCIA ENABLE TOKEN COUNTING , TRACCIA ENABLE COSTS , TRACCIA AUTO INSTRUMENT TOOLS , TRACCIA MAX TOOL SPANS , TRACCIA MAX SPAN DEPTH , TRACCIA OPENAI AGENTS , TRACCIA CREWAI , TRACCIA GUARDRAIL HEURISTICS Rate Limiting : TRACCIA MAX SPANS PER SECOND , TRACCIA MAX QUEUE SIZE , TRACCIA MAX BLOCK MS , TRACCIA MAX EXPORT BATCH SIZE , TRACCIA SCHEDULE DELAY MILLIS Runtime : TRACCIA SESSION ID , TRACCIA USER ID , TRACCIA TENANT ID , TRACCIA PROJECT ID , TRACCIA AGENT ID , TRACCIA AGENT NAME , TRACCIA ENV Legacy alias: TRACCIA PROJECT maps to project id Logging : TRACCIA DEBUG , TRACCIA ENABLE SPAN LOGGING Advanced : TRACCIA ATTR TRUNCATION LIMIT python from traccia import init Override config programmatically including agent identity for single-agent services init endpoint="http://tempo:4318/v1/traces", sample rate=0.5, enable costs=True, max spans per second=100.0, agent id="my-agent", agent name="My Agent", env="production", For services that orchestrate many logical agents in one process, set a service role and scope per-run identity: python from traccia import init, runtime config init service name="my-multi-agent-api", service role="orchestrator", auto start trace=False, with runtime config.run identity agent id="billing-agent", agent name="Billing Agent", env="production" : run one logical agent task ... This prevents the host service from being registered as a synthetic agent in the Traccia platform. If one process runs many agents concurrently for example, an API server or orchestrator , use this pattern: - Call init once per process for example at startup , not per request. - Wrap each logical "run" in runtime config.run identity ... to set agent id/name/env for that run. - Do not call stop tracing per request; use force flush to flush spans/metrics after a run without shutting down the provider. python from traccia import init, span, force flush, runtime config init service name="multi-agent-service", auto start trace=False def run agent agent id: str, env: str, payload: dict : Scope identity to this run only with runtime config.run identity agent id=agent id, agent name=agent id, env=env : with span "agent.run" as root: root.set attribute "agent.id", agent id root.set attribute "agent.run.mode", "api" ... your agent logic here ... Flush without tearing down the global provider force flush 5.0 The @observe decorator is the primary way to instrument your code: python from traccia import observe Basic usage @observe def process data data : return transform data Custom span name @observe name="data pipeline" def process data data : return transform data Add custom attributes @observe attributes={"version": "2.0", "env": "prod"} def process data data : return transform data Specify span type @observe as type="llm" "span", "llm", "tool" def call llm : pass Skip capturing specific arguments @observe skip args= "password", "secret" def authenticate username, password : pass Skip capturing result for large returns @observe skip result=True def fetch large dataset : return huge data Available Parameters : - name str, optional : Custom span name defaults to function name - attributes dict, optional : Initial span attributes - as type str : Span type - "span" , "llm" , "tool" , or "guardrail" - skip args list, optional : List of argument names to skip capturing - skip result bool : Skip capturing the return value @observe works seamlessly with async functions: python @observe async def async task x : await asyncio.sleep 1 return x 2 result = await async task 5 For more control, create spans manually: python from traccia import get tracer, span Using convenience function with span "operation name" as s: s.set attribute "key", "value" s.add event "checkpoint reached" do work Using tracer directly tracer = get tracer "my service" with tracer.start as current span "operation" as s: s.set attribute "user id", 123 do work Traccia automatically captures and records errors: python @observe def failing function : raise ValueError "Something went wrong" Span will contain: - error.type: "ValueError" - error.message: "Something went wrong" - error.stack trace: truncated stack trace - span status: ERROR Spans are automatically nested based on call hierarchy: python @observe def parent operation : child operation return "done" @observe def child operation : grandchild operation @observe def grandchild operation : pass Creates nested span hierarchy: parent operation └── child operation └── grandchild operation Traccia includes a powerful CLI for configuration and diagnostics: Create a new traccia.toml configuration file: traccia config init traccia config init --force Overwrite existing Validate configuration and diagnose issues: traccia doctor Output: 🩺 Running Traccia configuration diagnostics... ✅ Found config file: ./traccia.toml ✅ Configuration is valid 📊 Configuration summary: • API Key: ❌ Not set optional • Endpoint: https://api.traccia.ai/v2/traces • Sample Rate: 1.0 • OTLP Exporter: ✅ Enabled Test connectivity to your exporter endpoint: traccia check traccia check --endpoint http://tempo:4318/v1/traces Manage the local LLM pricing snapshot used for cost estimation: Show current pricing source, age, and model count traccia pricing status Download the latest pricing — tries the Traccia platform first, automatically falls back to the upstream pricing source if the platform is unreachable e.g. no Traccia account, network error traccia pricing refresh Force-fetch directly from the upstream pricing source skips the platform traccia pricing refresh --source upstream Remove local cache — reverts to the bundled snapshot shipped with the SDK traccia pricing clear Traccia computes estimated LLM costs locally at span-end time using a pricing table that resolves in the following order highest precedence first : | Priority | Source | Set via | |---|---|---| | 1 | Programmatic override | start tracing pricing override={...} | | 2 | Env var override | TRACCIA PRICING OVERRIDE JSON='{"model": {"prompt": x, "completion": y}}' | | 3 | Local cache | traccia pricing refresh stored in ~/.cache/traccia/pricing.json | | 4 | Bundled snapshot | Ships inside the SDK wheel generated from LiteLLM at release time | The bundled snapshot is generated at SDK release time from an upstream pricing database 2500+ models . It is refreshed automatically on every SDK release via the CI/CD pipeline — so pip install --upgrade traccia always brings a reasonably current snapshot. Between releases, you can pull the latest pricing without upgrading the SDK: traccia pricing refresh platform → upstream fallback The SDK warns you when the active snapshot is outdated: - 7 days old: logger.info — reminder to refresh - 30 days old: logger.warning — includes the exact command Every span also carries these attributes so the platform can always see how fresh the SDK's pricing was at emit time: | Attribute | Description | |---|---| | llm.pricing.generated at | ISO timestamp of the pricing snapshot | | llm.pricing.age days | Integer age in days at emit time | | llm.pricing.source | One of bundled \| local cache \| env \| override | If you use the Traccia platform, it maintains its own pricing sync every 6 hours by default . Costs shown in the platform UI are recomputed using the platform's authoritative snapshot, which is always fresher than the SDK's bundled table: org override platform snapshot bundled fallback When the platform recomputes a cost, it writes platform cost usd alongside the original llm.cost.usd on each span — the SDK-reported value is never overwritten . In the span detail panel you will see both values clearly labeled: - "Traccia recomputed cost" — what the platform computed, with a tooltip showing whether it came from an automatic pricing update or an org override. - "SDK-side pricing context at emit time " — the llm.pricing. attributes, collapsed by default, showing what prices the SDK saw when the span was emitted. Use traccia pricing refresh to sync the platform's snapshot into your SDK's local cache — then llm.cost.usd and platform cost usd will converge. You may occasionally see a muted "SDK reported $X.XX" line on the costs page. This happens when the two sources used different rates, for three legitimate reasons: | Reason | How to resolve | |---|---| | SDK's local cache or bundled snapshot is slightly older than the platform's current snapshot | Temporary — platform catches up within 6 hours, or click "Sync now" in Settings → Pricing. Running traccia pricing refresh also syncs the SDK. | | SDK-level override set via pricing override=... or TRACCIA PRICING OVERRIDE JSON | Intentional — the SDK override applies only to llm.cost.usd for that process. The platform uses its own snapshot + org overrides for platform cost usd . | | Org override set in Settings → Pricing on the platform | Intentional — org overrides apply to platform cost usd for all agents in the org. They do not change what SDK processes report. | The platform never uses the SDK's local cache as input to its own recomputation. Each agent process may have a different cached version, and the platform needs one consistent source for cross-agent reporting. SDK pricing affects what gets emitted in spans; platform pricing determines what gets shown in aggregate dashboards. All pricing overrides — whether set programmatically, via environment variable, or via the Traccia platform settings — use the same JSON shape: { "<model-name ": { "prompt": 0.005, "completion": 0.015, "cache write": 0.001, "cached prompt": 0.0005 } } All rates are USD per 1,000 tokens . cache write and cached prompt are optional for models with prompt-caching pricing, e.g. Claude 3.5 . Overrides always win over any other source: Programmatic per-process — both span attributes and cost metrics will use this traccia.start tracing pricing override={ "gpt-4o": {"prompt": 0.005, "completion": 0.015}, "my-fine-tuned-model": {"prompt": 0.01, "completion": 0.02}, } Environment variable persistent across restarts export TRACCIA PRICING OVERRIDE JSON='{"gpt-4o": {"prompt": 0.005, "completion": 0.015}}' Deprecation notice: AGENT DASHBOARD PRICING JSON is accepted as a back-compat alias for TRACCIA PRICING OVERRIDE JSON but will be removed in a future minor version. Rename the variable in your environment. Platform overrides org-level : Org admins can set pricing overrides in Settings → Pricing on the Traccia platform. These apply to the platform-recomputed cost platform cost usd for all agents in the org. They do not change llm.cost.usd on existing spans retroactively unless you explicitly enable the "Also recompute past traces" option in the save dialog. Protect your infrastructure with built-in rate limiting: rate limiting max spans per second = 100.0 Limit to 100 spans/sec max queue size = 5000 Max buffered spans max block ms = 100 Block up to 100ms before dropping Behavior : 1. Try to acquire capacity immediately 2. If unavailable, block for up to max block ms 3. If still unavailable, drop span and log warning When spans are dropped due to rate limiting, warnings are logged to help you monitor and adjust limits. Control trace volume with sampling: Sample 10% of traces init sample rate=0.1 Sampling is applied at trace creation time Traces are either fully included or fully excluded Automatic for supported LLM providers OpenAI, Anthropic : python @observe as type="llm" def call openai prompt : response = client.chat.completions.create model="gpt-4", messages= {"role": "user", "content": prompt} return response.choices 0 .message.content Span automatically includes: - llm.token.prompt tokens - llm.token.completion tokens - llm.token.total tokens - llm.cost.total in USD Traccia emits OTEL-compliant metrics for accurate cost and token tracking, independent of trace sampling. With trace sampling e.g., sample rate=0.1 , only 10% of traces are exported. Cost calculated from traces will be 10x underestimated . Metrics solve this by recording data for every LLM call, regardless of sampling. Traccia automatically emits these metrics: | Metric | Type | Unit | Description | |---|---|---|---| | gen ai.client.token.usage | Histogram | {token} | Input/output tokens per call | | gen ai.client.operation.duration | Histogram | s | LLM operation duration | | gen ai.client.operation.cost | Histogram | usd | Cost per call USD | | gen ai.client.completions.exceptions | Counter | 1 | Exception count | | gen ai.agent.runs | Counter | 1 | Agent runs CrewAI, OpenAI Agents | | gen ai.agent.turns | Counter | 1 | Agent turns | | gen ai.agent.execution time | Histogram | s | Agent execution time | Attributes : gen ai.system openai, anthropic , gen ai.request.model , gen ai.agent.id , gen ai.agent.name python from traccia import init init enable metrics=True, Default: True metrics endpoint="https://your-backend.com/v2/metrics", metrics sample rate=1.0, Default: 1.0 100% Or via traccia.toml : metrics enable metrics = true metrics endpoint = "https://your-backend.com/v2/metrics" metrics sample rate = 1.0 Or via environment variables: export TRACCIA ENABLE METRICS=true export TRACCIA METRICS ENDPOINT=https://your-backend.com/v2/metrics export TRACCIA METRICS SAMPLE RATE=1.0 Record your own metrics: python from traccia.metrics import record counter, record histogram Record a counter record counter "my custom events", 1, {"event type": "user action"} Record a histogram record histogram "my custom latency", 0.123, {"service": "api"}, unit="s" Agent-level metrics such as gen ai.agent.runs and gen ai.agent.execution time are only emitted when Traccia can see a real agent lifecycle for example, CrewAI crews or OpenAI Agents SDK runs . For plain OpenAI/Anthropic calls and most simple LangChain usages, you will still get full LLM metrics gen ai.client. , but no agent metrics unless you build an explicit agent abstraction on top. python import logging logging.basicConfig level=logging.DEBUG Or via config init debug=True Or via env var TRACCIA DEBUG=1 python your script.py 1. Check connectivity: traccia check 2. Validate config: traccia doctor 3. Enable debug logging 4. Verify endpoint is correct and accessible - Reduce max queue size in rate limiting config - Lower sample rate to reduce volume - Enable rate limiting with max spans per second - Check rate limiter logs for warnings - Increase max spans per second if set - Increase max queue size if spans are queued - Check traccia doctor output Initialize the Traccia SDK. All parameters are optional; configuration is merged from traccia.toml → env vars → explicit parameters highest wins . Parameters : Tracing - endpoint str : OTLP endpoint URL default: https://api.traccia.ai/v2/traces - api key str : API key for the Traccia platform - sample rate float : Sampling rate 0.0–1.0 default: 1.0 - auto start trace bool : Auto-start a root trace on init default: True - auto trace name str : Name for the auto-started trace default: "root" - use otlp bool : Use OTLP exporter default: True - service name str : Service name auto-detected if not set - service role str : "orchestrator" to prevent this service being registered as an agent - config file str : Path to a custom traccia.toml Exporters - enable console exporter bool : Print spans to stdout default: False - enable file exporter bool : Write spans to file default: False - file exporter path str : Path for file exporter default: "traces.jsonl" - reset trace file bool : Clear file on init default: False Instrumentation - enable patching bool : Auto-patch OpenAI, Anthropic, requests default: True - enable token counting bool : Count tokens default: True - enable costs bool : Calculate costs default: True - pricing override dict : Per-model pricing override — always wins over all other sources. See Pricing pricing section below. - openai agents bool : Auto-enable OpenAI Agents SDK integration default: True - crewai bool : Auto-enable CrewAI integration default: True - guardrail heuristics bool : Enable Tier C heuristic guardrail detection default: True - auto instrument tools bool : Experimental tool auto-instrumentation default: False - max tool spans int : Max tool spans per trace default: 100 - max span depth int : Max nested span depth default: 10 Agent identity single-agent services - agent id str : Logical agent identifier - agent name str : Human-readable agent name - env str : Deployment environment, e.g. "production" , "staging" Runtime metadata - session id str : Session identifier - user id str : User identifier - tenant id str : Tenant / org identifier - project id str : Project identifier Metrics - enable metrics bool : Emit OTEL metrics default: True - metrics endpoint str : Metrics endpoint derived from tracing endpoint if not set - metrics sample rate float : Metrics sampling rate default: 1.0 Rate limiting - max spans per second float : Rate limit spans/sec default: None = unlimited - max block ms int : Max ms to block before dropping a span default: 100 - max queue size int : Max buffered spans default: 5000 - max export batch size int : Spans per export batch default: 512 - schedule delay millis int : Batch export interval ms default: 5000 Misc - debug bool : Enable debug logging default: False - attr truncation limit int : Max attribute value length default: None Returns : TracerProvider instance Fetch a versioned prompt from Traccia cache + SWR + optional fallback . See Prompts in the SDK https://traccia.ai/docs/sdk/prompts . python from traccia import load prompt, prefetch prompts prefetch prompts "support-reply" prompt = load prompt "support-reply", label="production", fallback={...} messages = prompt.compile question=q evaluate name, , data, task, scorers=None, prompt=None, max concurrency=10, persist=True, provider keys=None, progress=True, ... - EvaluateResult Run a task and scorers over a platform dataset or inline rows. Persists an experiment by default. See Evaluate in the SDK https://traccia.ai/docs/sdk/evaluate . python from traccia import evaluate result = evaluate "support-reply-v3", data="support-golden", task=task, scorers= "exact match", "contains" , prompt="support-reply", persist=True, print result.summary print result.url Parameters : - name str : Experiment name required - data : Dataset name/UUID, or a list of {input, expected, metadata} dicts expected output is an alias of expected - task : Callable mapping each row to an output. First arg named row / item / example / case receives the full row; otherwise the input dict - scorers : Builtin ids exact match , contains , json valid , platform names/ids, and/or callables input=, output=, expected=, metadata= - prompt str, optional : Prompt name. Attaches version ids when persisting and labels the cell otherwise Task - max concurrency int : Parallel item workers default 10 - persist bool : Create an experiment default True . Inline + persist creates sdk-eval/<name /<id - provider keys dict, optional : BYO keys for platform LLM-as-judge openai , anthropic , gemini , groq and typesafe for Jev Decision. A Jev score keeps each question, so the experiment shows one chip per question. - progress bool : Print N/M to stderr default True Returns : EvaluateResult with rows , aggregates , summary , url , experiment id , dataset id , errors , persist error . Empty data, bad config, and dataset/scorer API failures raise EvaluateError . Stop tracing and flush pending spans. Parameters : - flush timeout float : Max seconds to wait for flush Get a tracer instance. Parameters : - name str : Tracer name typically module/service name Returns : Tracer instance Create a span context manager. Parameters : - name str : Span name - attributes dict, optional : Initial attributes Returns : Span context manager @observe name=None, , attributes=None, tags=None, as type="span", skip args=None, skip result=False Decorate a function to create spans automatically. Parameters : - name str, optional : Span name default: function name - attributes dict, optional : Initial attributes - tags list str , optional : User-defined identifiers for the observed method - as type str : Span type "span" , "llm" , "tool" , "guardrail" - skip args list, optional : Arguments to skip capturing - skip result bool : Skip capturing return value Load and validate configuration. Parameters : - config file str, optional : Path to config file - overrides dict, optional : Override values Returns : Validated TracciaConfig instance Raises : ConfigError if invalid Validate configuration without loading. Returns : Tuple of is valid, message, config or none Application Code @observe ↓ Span Creation ↓ Processors token counting, cost, enrichment ↓ Rate Limiter optional ↓ Batch Processor buffering ↓ Exporter OTLP/Console/File ↓ Backend Grafana Tempo / Jaeger / Zipkin / etc. - traccia.instrumentation. : Infrastructure and vendor instrumentation. - HTTP client/server helpers including FastAPI middleware . - Vendor SDK hooks and monkey patching e.g., OpenAI, Anthropic, Gemini, requests . - Decorators and utilities used for auto-instrumenting arbitrary functions. - traccia.integrations. : AI/agent framework integrations. - Adapters that plug into higher-level frameworks via their official extension points e.g., LangChain callbacks . - Work at the level of chains, tools, agents, and workflows rather than raw HTTP or SDK calls. Contributions are welcome Whether it's bug fixes, new features, documentation improvements, or examples - we appreciate your help. 1. Fork the repository 2. Create a feature branch : git checkout -b feature/amazing-feature 3. Make your changes and add tests 4. Run tests : pytest tests/ 5. Lint your code : ruff check traccia/ 6. Commit : git commit -m "Add amazing feature" 7. Push : git push origin feature/amazing-feature 8. Open a Pull Request Clone the repository Python SDK git clone https://github.com/traccia-ai/traccia-py.git cd traccia-py Create virtual environment python -m venv venv source venv/bin/activate On Windows: venv\Scripts\activate Install in editable mode with dev dependencies pip install -e ". dev " Run tests pytest tests/ -v Run with coverage pytest tests/ --cov=traccia --cov-report=html - Follow PEP 8 - Use type hints where appropriate - Add docstrings for public APIs - Write tests for new features - Keep PRs focused and atomic - Integrations : Add support for more LLM providers Cohere, AI21, local models - Backends : Test and document setup with different OTLP backends - Examples : Real-world examples of agent instrumentation - Documentation : Tutorials, guides, video walkthroughs - Performance : Optimize hot paths, reduce overhead - Testing : Improve test coverage, add integration tests Traccia adds GovernanceEvent attributes on spans automatically event type, integrity hash, timestamp source . No EU-specific setup is required for default tracing. Human review approve/reject, audit trail is done in the Governance Hub on the platform — not in the SDK. | Decorator | Purpose | Requires Traccia platform | |---|---|---| | @observe | Observability only — creates trace spans | No works with any OTLP backend | | @govern | Observability plus runtime policy enforcement | Yes | @govern does two things: 1. Checks agent status before the function body can warn or stop the next run for after-ingest policies . 2. Turns on a per-call check for this LLM or tool call Spend Cap, Model Boundary, Loop Cap . You do not call check policy yourself when using an instrumented LLM client and @observe as type="tool" . Tracing-only or self-hosted users should use @observe . python from openai import OpenAI from traccia import init, observe, govern, AgentBlockedError init api key="...", endpoint="https://api.traccia.ai/v2/traces", agent id="my-agent" client = OpenAI @observe name="lookup order", as type="tool" def lookup order order id: str - dict: return {"order id": order id, "status": "shipped"} @govern name="run agent" def run agent prompt: str - str: lookup order "ORD-1" resp = client.chat.completions.create model="gpt-4o", messages= {"role": "user", "content": prompt} , return resp.choices 0 .message.content or "" try: run agent "hello" except AgentBlockedError as exc: print exc.reasons, exc.decision id, exc.remaining budget usd Identity comes from init agent id=... or TRACCIA AGENT ID . Pass agent id on @govern only to override in a multi-agent process. Without an agent ID, API key or endpoint, @govern raises GovernanceConfigError a subclass of AgentBlockedError instead of running unchecked. On Block deny, @govern raises AgentBlockedError . Observe and Warn still let the call proceed and record a match. Refund Guard and Purchase Guard can hold a middle amount: @govern raises ApprovalPending , which is not AgentBlockedError . Catch it, do not run the tool, and do not retry the check. pending tool result is a normal tool result when a framework retries raised errors. For a custom tool that is not wrapped with @observe as type="tool" , call check policy action={"type": "tool call", "name": "refund"}, context={"input": {"amount": amount}} yourself. If a check gets no answer timeout, connection error, 5xx, or an invalid answer , the call follows your policy fail mode : Settings → Policy Enforcement in the app, for the organization or one workspace. The default is to block: the call raises AgentBlockedError , and the provider or tool is never called. Choose Allow there to let calls through while Traccia is down. - The setting arrives with answers the SDK already gets: the agent status at the start of each @govern run cached 60 s and each policy check. There are no extra requests, and tracing-only code never calls Traccia for it. - It is kept for 10 minutes after the last answer that carried it, then the SDK goes back to blocking. A process that has had no answer yet, for example one that starts while Traccia is down, blocks. Calling init again forgets it. - A local override applies only when there is no answer: init policy fail mode="open" | "closed" for the process, or @govern policy fail mode=... for one run and the checks inside it. It never overrides a rejected API key 401/403 , a block, a deny or a configuration error. The SDK logs a warning once when it disagrees with your Traccia setting, since the Traccia Gateway https://traccia.ai/docs/platform/gateway follows the setting. - @govern fail open=... is deprecated: True means policy fail mode="open" , False means "closed" . Omit it to follow your setting. Policies in the app: Policies https://traccia.ai/docs/platform/policies . SDK guide: Governance in the SDK https://traccia.ai/docs/sdk/governance . Advanced optional : override endpoints or cache TTL in traccia.toml : governance status check endpoint = "https://custom.example/agents/{agent id}/status" post block endpoint = "https://custom.example/agents/{agent id}/blocks" status cache ttl seconds = 120 policy fail mode = "closed" optional local override, see above Or pass status check endpoint , post block endpoint , status cache ttl seconds , or policy fail mode to init . Queue and decide reviews in Governance Hub → Reviews using a trace ID . The SDK does not create review queue rows. Use the dashboard when a person must sign off before you rely on an automated output. When your product shows the user they are interacting with AI banner, label, chat notice or synthetic content, call disclosure on the active span so exported traces carry audit evidence governance.transparency.disclosed , governance.event type=transparency . It does not render UI. python from traccia.governance import disclosure disclosure channel="ui", disclosed to user=True after user saw your AI notice init compliance={"frameworks": "eu ai act" , "risk tier": "high"} stamps eu ai act.risk tier on every span from that process. Static SDK config — not read from the dashboard registry. Link registry + agent in the UI for exports; set risk tier in init if you need it on every span. Best-effort regex masking email, phone, SSN-like, MRN/NPI/DOB heuristics . Not a guarantee that all PHI is removed — not ML/medical NER. python from opentelemetry import trace from traccia import init, observe from traccia.processors.redaction processor import redact string, apply redaction to span Automatic: redact sensitive attribute keys on every span before export init redact pii=True or TRACCIA REDACT PII=1 When compliance includes hipaa, redact pii defaults on unless explicitly set False init compliance={"frameworks": "hipaa" }, redact pii=True @observe def agent step user message: str - str: span = trace.get current span span.set attribute "gen ai.prompt", redact string user message manual out = run llm user message apply redaction to span span or rely on init redact pii=True return out - Platform : Governance Hub https://traccia.ai/docs/platform/governance — registry, reviews, incidents, evidence exports. - EU AI Act guide : docs/compliance/eu-ai-act https://traccia.ai/docs/compliance/eu-ai-act opt-in module; decision support only . - HIPAA guide : docs/compliance/hipaa https://traccia.ai/docs/compliance/hipaa opt-in module; soft PHI warnings; no signed BAA yet — contact support@traccia.ai mailto:support@traccia.ai . - Trust Center : trust-center https://traccia.ai/trust-center HIPAA shared responsibility . Apache License 2.0 - see LICENSE https://github.com/traccia-ai/traccia-py/blob/main/LICENSE for full terms and conditions. Built with: - OpenTelemetry https://opentelemetry.io/ - Vendor-neutral observability framework - Pydantic https://pydantic.dev/ - Data validation - tiktoken https://github.com/openai/tiktoken - Token counting Inspired by observability tools in the ecosystem and designed to work seamlessly with the OTLP standard. - Issues : GitHub Issues https://github.com/traccia-ai/traccia-py/issues - Report bugs or request features - Discussions : GitHub Discussions https://github.com/traccia-ai/traccia-py/discussions - Ask questions, share ideas Made with ❤️ for the AI agent community