Trace Multi-Agent LLM Pipelines with Langfuse and OpenTelemetry

Langfuse 4.14.4 and OpenTelemetry can be used to trace a two-agent Claude pipeline, capturing every LLM call, tool invocation, token count, and dollar cost in a single nested trace timeline. The tutorial, by Priya Nair, demonstrates instrumenting a research agent and a writer agent using the Langfuse v4 SDK and OpenLLMetry's AnthropicInstrumentor 0.62.3, with the Langfuse SDK acting as an OTel tracer provider to automatically collect spans. The setup requires Python 3.10+, a Langfuse account, and an Anthropic API key, with the LANGFUSE_BASE_URL env var matching the signup region to avoid silent trace loss.

Trace Multi-Agent LLM Pipelines with Langfuse and OpenTelemetry Instrument a two-agent Claude pipeline so every LLM call, tool hop, and token cost lands in one trace. Priya Nair https://sourcefeed.dev/u/priya nair What you'll build A two-agent LLM pipeline a research agent that calls a tool, then a writer agent fully instrumented with Langfuse https://langfuse.com over OpenTelemetry https://opentelemetry.io , so every LLM call, tool invocation, token count, and dollar cost shows up as a nested span in one queryable trace timeline. Prerequisites Python 3.10+ both langfuse and the instrumentor require ≥3.10 . Verified with Python 3.12. Package versions verified for this tutorial: langfuse 4.14.4 the v4 SDK, rewritten on OpenTelemetry — note the env var is now LANGFUSE BASE URL , not v3's LANGFUSE HOST , opentelemetry-instrumentation-anthropic 0.62.3 from OpenLLMetry https://github.com/traceloop/openllmetry , and a recentSDK 0.116+ . anthropic - A Langfuse account — the Cloud free tier https://cloud.langfuse.com works; note whether you signed up in the EU or US region. Self-hosted works identically, just point LANGFUSE BASE URL at your instance. - An Anthropic API key from the Claude Console https://platform.claude.com . Any OTel-instrumented provider works the same way; this tutorial uses Claude. - OS: anything with a shell. Commands below are macOS/Linux; on Windows use set instead of export . 1. Create a Langfuse project and grab keys Sign in at cloud.langfuse.com https://cloud.langfuse.com or us.cloud.langfuse.com for the US region , create an organization and a project, then go to Project Settings → API Keys → Create new API keys . You get a public key pk-lf-... and a secret key sk-lf-... . The secret is shown once — copy both now. 2. Install dependencies and set environment variables python -m venv .venv && source .venv/bin/activate pip install "langfuse =4.14" "opentelemetry-instrumentation-anthropic =0.62" anthropic export LANGFUSE PUBLIC KEY="pk-lf-..." export LANGFUSE SECRET KEY="sk-lf-..." export LANGFUSE BASE URL="https://cloud.langfuse.com" or https://us.cloud.langfuse.com export ANTHROPIC API KEY="sk-ant-..." LANGFUSE BASE URL must match the region you signed up in — this is the single most common source of silent trace loss. 3. Understand the two instrumentation layers You're wiring together two things, and it helps to know who does what: OpenLLMetry monkey-patches the Anthropic SDK and emits a standard OTel span for every API call, carrying AnthropicInstrumentor gen ai. semantic-convention attributes: model, prompt, completion, and token usage. The Langfuse v4 SDK is itself an OTel tracer provider. Any span emitted by any OTel instrumentation library while Langfuse is initialized lands inside your Langfuse trace tree automatically — no exporter config, no OTLP endpoint wrangling. Langfuse maps gen ai.usage. to token counts and multiplies them against its built-in model price list updated daily against provider docs to compute cost per generation. Your own agent and tool functions become spans via Langfuse's @observe decorator, and the auto-instrumented LLM spans nest under whichever @observe function made the call. 4. Build the instrumented pipeline Save this as pipeline.py . It's the complete, runnable file: python import os from anthropic import Anthropic from langfuse import get client, observe, propagate attributes from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor 1. Patch the Anthropic SDK BEFORE making any calls. AnthropicInstrumentor .instrument 2. Initialize Langfuse reads LANGFUSE env vars and fail fast on bad creds. langfuse = get client if not langfuse.auth check : raise SystemExit "Langfuse rejected credentials — check keys and LANGFUSE BASE URL region" client = Anthropic MODEL = "claude-opus-5" def ask system: str, prompt: str - str: Opus 5 thinks by default and max tokens caps thinking + answer together, so leave generous headroom. response = client.messages.create model=MODEL, max tokens=16000, system=system, messages= {"role": "user", "content": prompt} , return "".join b.text for b in response.content if b.type == "text" @observe tool invocation - its own span, input/output captured def fetch release notes project: str - str: Stub tool: swap in a real HTTP call or DB query. return f"{project} changelog: v4 SDK is OTel-native; ingestion adds " "x-langfuse-ingestion-version=4; cost table now audited daily." @observe name="research-agent" def research topic: str - str: notes = fetch release notes topic return ask "You are a research agent. Extract the three most important facts as bullets.", f"Source material:\n{notes}", @observe name="writer-agent" def write summary facts: str - str: return ask "You are a writing agent. Turn these facts into a two-sentence executive summary.", facts, @observe name="research-pipeline" def run pipeline topic: str - str: return write summary research topic if name == " main ": Attributes set here propagate to every span in the trace, so you can filter/group traces by session or user in the UI. with propagate attributes session id="demo-session-1", user id="tutorial-reader" : print run pipeline "Langfuse" langfuse.flush spans are buffered; short scripts must flush before exit Two details matter more than they look. AnthropicInstrumentor .instrument runs before anything else so every SDK call is patched. And langfuse.flush runs last — the SDK exports spans on a background thread, and a script that exits without flushing loses the trace. Privacy note: the instrumentor records prompts and completions into span attributes by default. Set TRACELOOP TRACE CONTENT=false to keep payloads out of your traces. 5. Run it python pipeline.py The script prints a two-sentence summary, e.g.: Langfuse's v4 SDK is now built natively on OpenTelemetry, with ingestion tagged via x-langfuse-ingestion-version=4. Its model cost table is audited daily, keeping per-generation pricing accurate. Verify it works Open your project in the Langfuse UI and click Tracing → Traces . You should see a trace named research-pipeline within a few seconds. Click it and check: The timeline nests correctly: research-pipeline → research-agent → fetch release notes span + one anthropic.chat generation , then writer-agent → a second generation. Two generations total. Each generation shows claude-opus-5 as the model, the full prompt and completion, input/output token counts, latency, and a USD cost computed from Langfuse's price table. Trace metadata shows user id: tutorial-reader and session id: demo-session-1 ; under Tracing → Sessions , demo-session-1 groups this run re-run the script and both traces appear in the session . If all three hold, every hop of the pipeline is now queryable — you can filter traces by session, sort by cost, and drill into any slow or expensive generation. Troubleshooting — nine times out of ten the keys are fine and auth check fails or the script exits with "Langfuse rejected credentials" LANGFUSE BASE URL points at the wrong region: US-region keys against https://cloud.langfuse.com the EU default return 401. Match the URL to where you created the project. Set LANGFUSE DEBUG=True to see the failing request. Script succeeds but no trace appears in the UI — the process exited before the background exporter drained its buffer. Ensure langfuse.flush or langfuse.shutdown runs at the end of the script, including on exception paths; in serverless handlers, flush inside the handler before returning.— the Anthropic key is missing, mistyped, or was revoked. Re-export anthropic.AuthenticationError: Error code: 401 ... 'invalid x-api-key' ANTHROPIC API KEY ; if you use a key manager, confirm the venv shell actually inherits it echo $ANTHROPIC API KEY . Generations show token counts but the cost column is blank — the model string on the span didn't match any Langfuse model definition common with brand-new or fine-tuned models . Add one under Project Settings → Models with a regex match pattern and per-token prices; custom definitions override built-ins and apply to new traces immediately. Next steps - Add scores https://langfuse.com/docs/evaluation/overview to traces user feedback, LLM-as-a-judge so you can correlate quality with cost per agent. - Instrument other layers of the stack — any OTel library HTTP clients, databases, other LLM SDKs drops its spans into the same trace; see Langfuse's OpenTelemetry docs https://langfuse.com/integrations/native/opentelemetry for the attribute mapping and the OTLP endpoint if you'd rather bring your own collector. - Use langfuse.start as current observation as type="generation", ... for manual control when auto-instrumentation doesn't fit — for example, custom retry loops or providers without an instrumentor. - Wire the same env vars into CI and production; traces are cheap enough to leave on everywhere, and the session/user attributes you propagated make per-tenant cost accounting a saved table view. Sources & further reading - Observability for Anthropic with Langfuse Integration https://langfuse.com/integrations/model-providers/anthropic — langfuse.com - Langfuse Python SDK Overview v4 https://langfuse.com/docs/observability/sdk/python/overview — langfuse.com - Langfuse Python SDK Instrumentation https://langfuse.com/docs/observability/sdk/python/instrumentation — langfuse.com - Token and Cost Tracking https://langfuse.com/docs/observability/features/token-and-cost-tracking — langfuse.com - OpenTelemetry OTLP Integration https://langfuse.com/integrations/native/opentelemetry — langfuse.com - opentelemetry-instrumentation-anthropic https://pypi.org/project/opentelemetry-instrumentation-anthropic/ — pypi.org Priya Nair https://sourcefeed.dev/u/priya nair · AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to. Discussion 0 No comments yet Be the first to weigh in.