Langfuse for LLM Observability: Tracing Agent Calls Instead of Guessing

A developer detailed how Langfuse provides the trace/span/generation hierarchy needed to debug multi-step agentic LLM workflows, contrasting it with observability tools that only capture top-level calls. The developer consolidated multiple tools onto Langfuse to reduce resource sprawl and attached evaluation scores directly to traces, and shared self-hosting guidance for Kubernetes using CloudNativePG.

An agent makes six tool calls, picks the wrong one on step four, and the final output is garbage. You stare at your logs. You see the input. You see the output. Everything in between is a void. That's the black box problem with agentic LLM workflows, and it's the reason I started looking at Langfuse seriously. If you're running multi-step agents LangChain, custom loops, or any orchestration layer , you need per-step tracing with enough context to reconstruct why the agent chose what it chose. Langfuse gives you that. But getting it wired up correctly, especially in a self-hosted Kubernetes environment alongside other observability tools, has a few sharp edges worth knowing about. Before I get into Langfuse itself, I want to talk about a failure mode I see constantly with LLM tooling: observability sprawl. Here's how it usually plays out. You spin up Dify because it has a nice agent builder. You add Opik because someone recommended it for evaluation. You deploy AnythingLLM for RAG experiments. Each tool has its own Postgres database, its own PVC, its own memory footprint. Before you know it, you've got three separate platforms that each capture some traces, and none of them give you the full picture. Resource costs compound quickly. In a homelab or small-cluster environment, those redundant tools can easily consume 8-10 GB of RAM and 50-70 GB of persistent storage. Those are real resources you're giving up for the privilege of having your debugging split across multiple dashboards. Pick one tool, instrument everything through it, and delete the rest. Langfuse is the one I picked, and the consolidation alone was worth it. But the reason I picked it over alternatives comes down to one specific feature. Most LLM observability tools trace at the wrong granularity. They capture the top-level call: here's the prompt, here's the completion, here's the token count. Fine for a single chat.completions call. Nearly useless for an agentic workflow. Consider what happens in a typical agent loop. An orchestrator receives a user query. It decides which tool to call. That tool might call an LLM itself for summarization, extraction, or routing . Results come back, and the orchestrator decides whether to call another tool or return a final answer. A single user request might involve four or five LLM calls, each with different prompts, different models, and different failure modes. What I needed was the ability to trace the full execution tree: one top-level "trace" for the user request, with nested "spans" for each agent step, and nested "generations" for each LLM call within those steps. Langfuse calls this the trace/span/generation hierarchy, and it maps cleanly onto how multi-agent systems https://guatulabs.dev/posts/multi-agent-ai-systems-architecture-patterns/ actually work. Evaluation scores were the other hard requirement, and I wanted them attached to traces, not living in a separate system. I had a custom evaluation layer built with Zod schemas that validated agent outputs against expected structures. It worked, but it was brittle, lived in application code, and had no dashboard. Langfuse lets you attach numeric scores to any trace or span, which means your evaluation data lives right next to your trace data. One place to look. Langfuse has a managed cloud offering, but if you're already running a cluster, self-hosting is straightforward. The project provides a Helm chart and Docker images. The main dependency is Postgres. If you're already running CloudNativePG https://guatulabs.dev/posts/cloudnativepg-running-postgresql-in-kubernetes-without-the-pain/ , you can point Langfuse at an existing cluster. Create a dedicated database for it: apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: langfuse-db namespace: observability spec: instances: 2 storage: size: 10Gi bootstrap: initdb: database: langfuse owner: langfuse Langfuse itself is a single container with environment variables for the database connection, a secret key, and your desired auth settings. A minimal Kubernetes deployment looks like this: apiVersion: apps/v1 kind: Deployment metadata: name: langfuse namespace: observability spec: replicas: 1 selector: matchLabels: app: langfuse template: spec: containers: - name: langfuse image: langfuse/langfuse:2.x ports: - containerPort: 3000 env: - name: DATABASE URL valueFrom: secretKeyRef: name: langfuse-db-credentials key: uri - name: NEXTAUTH SECRET valueFrom: secretKeyRef: name: langfuse-auth key: secret - name: NEXTAUTH URL value: "https://langfuse.example.com" - name: SALT valueFrom: secretKeyRef: name: langfuse-auth key: salt If you're deploying through ArgoCD https://guatulabs.dev/posts/gitops-for-homelabs-argocd-app-of-apps/ , there's a gotcha worth flagging. If you organize your observability stack in a directory structure say, observability/langfuse/ , observability/grafana/ , etc. and use a directory-type Application source, you need to set directory.recurse: true . Without it, ArgoCD will show "0 managed resources" even though your manifests exist in subdirectories. It's a silent failure that'll have you rechecking file paths for twenty minutes before you realize ArgoCD just isn't looking deep enough. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: observability spec: source: repoURL: https://git.example.com/infra.git path: observability directory: recurse: true without this, subdirectories are invisible Once Langfuse is running, the real work begins: instrumenting your agent code so each step shows up as a distinct span in the trace tree. Langfuse's Python SDK makes this fairly clean with decorators. A minimal example for a custom agent loop: python from langfuse.decorators import observe, langfuse context from openai import OpenAI client = OpenAI @observe as type="generation" def call llm prompt: str, model: str = "gpt-4o" - str: response = client.chat.completions.create model=model, messages= {"role": "user", "content": prompt} , return response.choices 0 .message.content @observe def search tool query: str - str: your tool logic here results = do search query return results @observe def agent loop user query: str - str: plan = call llm f"Plan steps for: {user query}" for step in parse steps plan : if step.tool == "search": result = search tool step.input elif step.tool == "summarize": result = call llm f"Summarize: {step.input}" each iteration creates a child span automatically final = call llm f"Final answer given results: {result}" return final Every function decorated with @observe becomes a span in Langfuse. Functions marked as type="generation" get special treatment: Langfuse records token counts, model name, latency, and prompt/completion pairs. Nested calls automatically create a parent-child hierarchy, so when you open a trace in the Langfuse UI, you see the full tree. For TypeScript/Node.js backends, the pattern is similar but uses the Langfuse client class directly: python import Langfuse from "langfuse"; const langfuse = new Langfuse { publicKey: process.env.LANGFUSE PUBLIC KEY, secretKey: process.env.LANGFUSE SECRET KEY, baseUrl: "https://langfuse.example.com", } ; async function tracedAgentCall userQuery: string { const trace = langfuse.trace { name: "agent-request" } ; const planSpan = trace.span { name: "planning" } ; const plan = await callLLM userQuery ; planSpan.update { output: plan } ; planSpan.end ; for const step of parseSteps plan { const toolSpan = trace.span { name: tool:${step.tool} , input: step.input, } ; const result = await executeTool step ; toolSpan.update { output: result } ; toolSpan.end ; } await langfuse.flushAsync ; } Notice the explicit flushAsync at the end. Langfuse batches events for performance. In serverless or short-lived processes, skipping the flush means you lose traces silently. I've seen this bite people running agents in Lambda functions or one-shot scripts. Before Langfuse, my evaluation layer was a hand-rolled mess. Zod schemas validated agent outputs, results got persisted to a JSON file or a database table, and "evaluation" meant grepping through structured logs. It worked, in the sense that a Rube Goldberg machine works. Langfuse replaces that with score calls attached directly to traces: python Before: custom evaluation persisted to database from zod validator import AgentOutputSchema import json def evaluate and persist output, expected schema : result = AgentOutputSchema.safeParse output with open "eval log.jsonl", "a" as f: json.dump { "valid": result.success, "errors": result.errors if not result.success else None, "timestamp": datetime.now .isoformat }, f f.write "\n" After: scores live in Langfuse alongside traces from langfuse.decorators import observe, langfuse context @observe def agent with eval user query: str - str: result = agent loop user query attach a quality score to this trace langfuse context.score current trace name="output valid", value=1.0 if validate output result else 0.0, attach a relevance score langfuse context.score current trace name="relevance", value=compute relevance user query, result , comment="cosine similarity against expected answer", return result Now your evaluation data shows up in the same dashboard as your traces. You can filter traces by score, spot regressions over time, and correlate low scores with specific agent steps that failed. No more cross-referencing JSON log files with application logs. Langfuse's trace/span/generation model maps onto agentic workflows because it mirrors the actual call stack. A trace is a complete user request. Spans are logical operations within that request. Generations are the individual LLM calls. This hierarchy means you can answer questions that flat logging can't: Compare this to what you get with Grafana dashboards https://guatulabs.dev/posts/grafana-dashboards-information-density-vs-readability/ . Grafana excels at aggregate metrics: request rate, p99 latency, error percentage. It shows you the forest. Langfuse shows you individual trees. You need both, but for debugging agent behavior, the per-trace detail is what saves you. Prompt management is another underappreciated feature. Langfuse lets you version prompts in its UI, then fetch them at runtime by name and version. This decouples prompt iteration from code deployment. Your prompt engineer or you, wearing that hat can tweak prompts and track how each version affects scores, without touching application code or triggering a redeploy. One thing to think about early: agent traces often contain sensitive data. Tool inputs might include search queries, user data, or service account credentials https://guatulabs.dev/posts/agent-credential-management-two-tier-service-accounts/ . Langfuse stores everything you send it. If you're self-hosting, this is manageable because the data stays in your cluster. But you should still be intentional about what gets logged. Scrub sensitive fields before they hit the trace: php @observe def safe tool call tool name: str, params: dict - str: sanitized = {k: v for k, v in params.items if k not in "api key", "token", "password" } langfuse context.update current observation input=sanitized, only safe fields return execute tool tool name, params full params for execution For managed Langfuse their cloud , check your data handling requirements before shipping traces that contain PII or internal API responses. If you're building AI agent services https://guatulabs.com/services for clients, this is a compliance conversation you want to have before the first trace lands. Consolidate early. Running multiple LLM observability tools feels productive because you're "evaluating options." In practice, it means your traces are fragmented, your resource usage balloons, and you debug slower because you're checking two dashboards for every issue. Pick one tool and commit. If Langfuse doesn't fit your stack, pick something else, but pick one . Instrument at the span level from day one. Adding tracing to an existing agent codebase after the fact is painful. Every function needs to be wrapped, and you inevitably miss the one tool call that turns out to be the problem. If you're building a new agent, add @observe decorators as you write each function. Retrofitting is always harder. Flush your traces. Langfuse batches events for efficiency, which means traces can be lost if your process exits before the batch ships. Call langfuse.flush Python or langfuse.flushAsync TypeScript at the end of every request handler. In serverless environments, this is not optional. Scores are cheap. Use them. Attaching a score call adds negligible overhead, but it gives you trend data you can't get any other way. Even a simple binary "output was valid" score, aggregated over hundreds of traces, tells you whether your agent is getting better or worse after a prompt change. I score every trace now, even if the scoring logic is basic. Self-hosting is worth it for sensitive workloads. Agent traces contain prompts, tool outputs, and sometimes user data. Keeping that data on your own cluster, behind your own network policies, is worth the operational overhead of managing a Postgres database and a single container. If you're already running Kubernetes with CloudNativePG, the marginal cost is low. Langfuse isn't the most exciting tool I've deployed. It doesn't generate flashy demos. But it's the tool that made my agent debugging go from "stare at logs and guess" to "open the trace, click the failing span, read the prompt." For anything running in production, that difference is everything.