Graph Database Product Specialist, Neo4j
10 min read
Your AI agent approved a refund outside the standard time frame. Now a compliance officer wants to know why, and whether the support agent granted a refund like this before. The chat log shows the conversation between the customer and the agent. The log doesn’t show what the agent checked before granting the refund or how it handled similar situations previously.
Without a decision trace, you can try to piece together why an agent made a decision, but its reasoning is really a black box. With a decision trace, you get a structured record of how an AI agent reached a decision. It captures not only the outcome but also the reasons and tools the agent used. This is stored in a context graph, where those steps stay connected to the conversations, facts, and other information the agent relied on, so you can follow them to the outcome.
Here’s why that matters, and what it takes to give your agents that kind of persistent, queryable memory.
Why AI agents need a record of their reasoning #
AI agents make many decisions as they work through a task. They use the information available to them, choose tools, apply policies, and decide what to do next. Because a nondeterministic LLM sits at the center of this process, the same input doesn’t always produce the same path or outcome, and once the run ends, the context behind those choices can disappear. Over a long workflow, an agent may also lose track of earlier decisions or drift from the original goal.
Decision traces give those decisions a durable record. You can inspect that history when an agent behaves unexpectedly or when another team needs to review how the agent handled a case. That record supports the governance and audit requirements for production systems.
Past traces can become part of the context for future tasks, too. Alongside enterprise knowledge and conversations, an agent can retrieve relevant past decisions as precedents, and humans can weigh in so that precedents reflect business needs and mistakes don’t get repeated.
Recording decision traces pays off every time a past decision gets looked at again:
- Explainability: Every decision stays connected to the information and previous decisions that shaped it, so you can trace the rationale behind it directly. For example, when that refund gets questioned later, the trace shows the policy the agent applied and the order details it checked.
- Debugging: Multi-step agent workflows can fail for many reasons. A decision trace shows each step, tool call, and result it recorded, making it easier to identify where a workflow went off track. The trace shows the exact step where a tool call came back empty, and the agent improvised.
- Compliance and auditing: Decision traces capture not just what the agent decided but how it got there, creating anaudit trail of every step, tool call, and policy the agent consulted. That record meets the transparency demands that regulators and stakeholders increasingly place on automated decisions.
- Consistency: By retrieving relevant decision traces, an agent applies the same approach to similar cases and produces more consistent outcomes. The next refund request with the same parameters retrieves the earlier trace as precedent, so the agent provides the same answer.
- Learning over time: Every completed trace adds another example to the graph. As the collection grows, agents can search previous decisions, find similar situations, and reuse approaches that worked well before. Reasoning memory records how decisions get made and why — a compounding loop of institutional intelligence.
- Shared reasoning across agents: When multiple agents use the same context graph, they can retrieve relevant decision traces from one another and build on prior work. A triage agent can read the resolution agent’s past traces before escalating a case, and each sees the decisions it took no part in making.
What does a decision trace record? #
Each decision trace provides the breadcrumbs you need to better understand what happened as an agent works through a decision. These include the following core elements:
- Decision: What the agent decided on.
- Outcome: The result of that decision.
- Reasoning steps: The rationale behind decisions.
- Tool calls: The tools the agent called, the actions it took, and the results it received.
- Context: The entities, conversations, and other information used at each step are linked back to the trace.
Take the support agent example. Before approving the refund, it checks the order and applies the relevant refund policy. The decision trace connects those steps and the policy lookup result to the final approval, so someone reviewing the case later can see what information the agent used and what it did before making the decision.
Decision traces vs. LLM traces vs. logs #
Decision traces, LLM traces, and application logs can all tell you something about an agent’s activity, but they answer different questions.
| Decision traces | LLM traces | Application logs | |
|---|---|---|---|
| Primary question | Why did the agent make this decision? | What happened during this model run? | What happened and when? |
| Records | Decisions, outcomes, reasoning steps, tool calls, and context | Model inputs, outputs, tool calls, and run details | Events, timestamps, errors, and system activity |
| Primary use | Explain, audit, and reuse past decisions | Debug and inspect model runs | Monitor and troubleshoot application behavior |
| Persistence | Reasoning memory that the agent can query and reuse | Scoped to a single model run and stored in an observability tool | Stored as an event history |
You use a log to see the basic actions, while an LLM trace like LangSmith can help you inspect the model that produced the actions, but only a decision trace connects the dots. The trace lets you walk back from the approval to the information, actions, and prior decisions that shaped it. And unlike the other two, it persists as memory that the agent itself can retrieve, inside the same graph as the knowledge and conversations the decision drew on.
Decision traces in a context graph #
Decision traces are primarily stored in a context graph, a persistent memory system for AI agents, where the connections between memories stay traversable. It brings together three types of memory. Long-term memory holds enterprise knowledge, short-term memory captures conversation history, and reasoning memory records decision traces. Together, these layers give an agent access to what it knows, what happened in the current interaction, and the decisions it made along the way.
Decision traces form the reasoning memory layer and connect directly to the other two types of memory. That means an outcome can connect directly to the company facts the agent relied on and the conversation that triggered it. Going back to that support agent scenario, that refund approval can connect not only to the customer’s request but also to the relevant refund policy and the other aspects of the decision trace. Following those relationships shows the full context around the decision.
Because the three memory layers share the same connected graph, an agent can query them together to see the full context.
How to record decision traces with Neo4j #
Neo4j Agent Memory is a memory system for single- and multi-agent systems that brings short-term, long-term, and reasoning memory together in a Neo4j graph.
It ships as Python and TypeScript SDKs backed by a hosted memory service, with a self-hosted option for teams already running Neo4j. It integrates with agent frameworks, including LangChain, Pydantic AI, LlamaIndex, CrewAI, and OpenAI Agents SDK, so you can add memory to agents you’ve already built. Neo4j also provides Model Context Protocol (MCP) tools for agent integrations.
Trace capture is explicit: Your code decides what gets recorded and when. In practice, you write that wiring once. With LangChain, for example, a callback handler opens a trace when a chain starts, records each tool call as it happens, and completes the trace when the chain ends. Every agent run after that is captured without further work.
The trace lifecycle is a simple block of code using the neo4j-agent-memory SDK. For Python, install it with pip install neo4j-agent-memory.
import asyncio
from pydantic import SecretStr
from neo4j_agent_memory import MemoryClient, MemorySettings
async def handle_refund():
settings = MemorySettings(
neo4j={"uri": "bolt://localhost:7687", "password": SecretStr("password")}
)
async with MemoryClient(settings) as memory:
trace = await memory.reasoning.start_trace(
"refund-4823",
task="Handle refund request #4823",
)
step = await memory.reasoning.add_step(
trace_id=trace.id,
thought="Check refund eligibility against policy",
)
await memory.reasoning.record_tool_call(
step.id,
tool_name="refund_policy_lookup",
arguments={"order_id": "4823"},
result={"eligible": True, "window_days": 30},
)
await memory.reasoning.complete_trace(
trace.id,
outcome="Refund approved",
success=True,
)
asyncio.run(handle_refund())
Neo4j Agent Memory records decision traces as the agent works. start_trace() opens a trace, add_step() adds each thought and action, and record_tool_call() captures tool calls and their results. When the task ends, complete_trace() closes the trace with the outcome.
Once recorded, those traces become part of the context graph, where you can inspect reasoning steps, tool calls, and their connections to the surrounding context.
Reading a trace is just as direct. get_trace() returns the complete record — the steps, tool calls, and outcome — and get_similar_traces() finds past decisions like the one at hand, so an agent or a reviewer can check precedent before acting.
full_trace = await memory.reasoning.get_trace(trace.id)
precedents = await memory.reasoning.get_similar_traces(
"Handle refund request", limit=3
)
That first call answers the compliance officer in our earlier support scenario: The trace shows the agent checked the order, looked up the refund policy, confirmed eligibility within the 30-day window, and approved the refund.
For a faster start, Create Context Graph can scaffold a full-stack agent application with a knowledge graph, decision traces, streaming chat, and graph visualization built in. If you want to go deeper, you can explore the Neo4j Agent Memory documentation and GraphAcademy course.
For a practical example, see how Lenny’s Memory uses a context graph to give an AI agent persistent memory.
Give your agents reasoning memory #
Decision traces give agents a record of the decisions they’ve made, the context behind them, and the resulting outcomes. As part of a context graph’s reasoning memory, those traces give you and other teams a way to inspect past decisions and why agents made them.
That history becomes more useful as it grows. Agents retrieve relevant past decisions when they encounter similar situations, while teams can use the same traces to explain decisions, review agent behavior, and support governance requirements. Combined with enterprise knowledge and conversation history, reasoning memory gives agents the full context they need to make consistent, explainable decisions.
Neo4j is the knowledge layer for enterprise AI — connecting the context, memory, and reasoning that agents rely on. The next time someone asks why an agent made a particular decision, the answer can be a graph query away.
Context graphs: Agent memory with Neo4j #
Take the free GraphAcademy course to learn how to build agent memory, including decision traces, with Neo4j.
Decision traces: FAQs #
Decision traces are structured records of how an AI agent reached a decision. Each one captures the decision itself, the reasons the agent recorded along the way, and the tools it used to reach the outcome.
AI agents reason through a task step by step, but once the run ends, much of the context behind those choices can disappear. Over a long workflow, an agent may also lose track of earlier decisions or drift from the original goal. Decision traces give those decisions a durable record for agents to act consistently.
A decision trace records the decision and its outcome, each reasoning step’s rationale and action, every tool call and its result, and links to the entities and conversation the decision drew on.
Logs capture what happened, while LLM traces record the details of a model run for debugging. Decision traces capture why an agent made a decision and persist as memory the agent can query and reuse in future tasks.
Before acting, an AI agent can search past decision traces for similar situations and retrieve relevant precedents. It can then reuse approaches that worked before, helping it handle similar cases more consistently and improve its decisions over time.
Neo4j Agent Memory records decision traces as an agent works. start_trace() opens a trace, add_step() and record_tool_call() capture the reasoning and tool use, and complete_trace() closes the trace with the outcome.