The moment arrives after the damage. An agent has cancelled a supplier contract, approved a refund it should have declined or moved a customer record somewhere it should never have gone. Someone senior asks a simple question. Why did it do that? The engineering team opens the logs. There are thousands of lines. Every call is timestamped. Every tool invocation is recorded. And nobody in the room can say why the agent chose what it chose.
This pattern is becoming familiar. It will become more so. Logging in most enterprises is mature. Security teams have spent two decades building collection pipelines, retention policies and search tooling. The natural assumption is that this investment transfers to AI agents. It does not. Logs capture events. Accountability requires decisions that can be rebuilt after the fact and rebuilding a decision needs four elements that most logging captures only in part.
An event log records that something happened and when. A decision record captures the basis for that decision. People often confuse the two because, in conventional software, the distinction rarely mattered. Code was deterministic. Anyone who knew the inputs and the version could rerun the logic and arrive at the same result. The log was enough because you could interrogate the system itself.
Agents break this assumption in four ways. First, their output varies. Running the same scenario twice can produce two different answers. Replay, the standard investigative move, no longer works.
Second, a single decision spans many steps. An agent that resolves a customer complaint might retrieve a policy, query an account, call a pricing tool, consult a second agent and then act. The log shows twelve entries. The decision was one.
Third, the system that made the decision may no longer exist. Models are updated. Prompts are revised. Tool configurations change weekly. Six months later, the agent that made the call has been replaced by something that behaves differently.
Fourth, authority is distributed. A human sets a goal. An orchestrating agent delegates to a specialist agent. The specialist acts. Each step is logged somewhere, usually in a different system, and nothing ties the chain together.
One clarification is needed before going further. This article is about reconstructing decisions, which is a records problem. It is separate from model interpretability, which asks why a neural network produced a particular output. Interpretability is a research field with partial answers. Reconstruction is a governance discipline with practical answers. Leaders should keep the two separate, because solving the second doesn’t depend on solving the first.
Regulation is already moving on the records problem. The EU AI Act requires high-risk systems to keep automatic logs sufficient to trace their operation, and it gives affected individuals a right to an explanation of decisions that significantly affect them. Explanation presupposes reconstruction. A log that cannot be turned into an account of the decision will satisfy neither obligation.
If logs leave a gap, the next question is what fills it. Four elements do. Every decision, human or machine, can be rebuilt from four kinds of fact: what the decider knew, how it chose, who allowed it to act and what changed as a result. Each fact about an agent’s decision belongs to exactly one of these. Together they cover everything a board, an auditor or a regulator will ask.
This element covers the inputs present at the moment of decision. The user request. The documents retrieved. The results returned by each tool. The model version, prompt version and configuration in force.
Typical logs record references and leave out contents. They note that a document was retrieved and store its identifier. They note that a tool was called and store the call. The document is later edited. The tool’s underlying data moves on. The reference survives, but the substance it pointed to is gone. An investigator can prove the agent looked at something and cannot show what it saw.
The test question: could the team produce the exact information an agent acted on six months ago, as it stood at that moment?
This element covers the reasoning applied. The objective the agent was pursuing. The policy or rule it applied. The thresholds that triggered the action. The alternatives it considered and rejected. Its stated confidence, if any.
Logs record the selected action and discard everything else. The agent weighed three options and chose one. The log holds the one. Any explanation after the fact has to guess at the other two, and guessing is where credibility fails under questioning. For agents that generate intermediate reasoning, capturing it is possible and often cheap. For those that do not, the policy and objective can still be recorded, and they explain a great deal.
The test question: can the team state which rule, goal or threshold produced this specific outcome?
This element covers the chain of permission. The scope delegated to the agent. The approval checkpoints it passed or bypassed. The handoffs between agents in a multi-agent system. The named human who owns the outcome.
The gap here is structural. Permissions live in an identity or configuration system. Actions live in an application log. The two are rarely joined. When an agent acts outside its intended scope, asking “was this allowed?” requires manual reconciliation between systems never designed to be read together. In multi-agent arrangements, the problem compounds, because the orchestrator’s delegation and the specialist’s execution are logged in different places with different identifiers.
Authority is also the element most likely to be asked about first. Regulators and boards want a name. “The agent did it” is an answer that satisfies nobody.
The test question: can the team name the individual accountable for this decision, and show the delegation that connects them to the action?
This element covers the effects. The systems the action touched. The downstream decisions it triggered, by other agents or by people. Whether it can be reversed, and whether it was.
Logs stop at the agent’s boundary. The agent sent an instruction to a payments system, and the payments system has its own log. The agent flagged a case, and a human then acted on the flag without any record linking the two. Effects propagate. The record stays behind.
Consequence also determines how much of the other three elements is worth keeping. A decision that moves money or changes a customer’s legal position warrants a complete record. A decision to reformat a document does not.
The test question: can every downstream action be traced back to the decision that caused it?
Diagnosis is useful only if it changes practice. Three actions close the gap.
Taken together, these three actions form the beginnings of what I call the Decision Layer: the records, roles and routines that make any consequential decision explainable long after it was made. Agents make the need for it impossible to postpone.
Return to the room. The agent has acted. The senior leader asks why. This time the team opens a decision record. It shows what the agent knew, the policy it applied, the manager who delegated the scope and the two downstream systems the action touched. The explanation takes ten minutes. The remediation starts the same afternoon. That outcome is a design choice. Logging is necessary, and most enterprises already have it. Reconstruction is the harder discipline, and accountability depends on it. The question for every board with agents in production is simple. If an agent’s decision were challenged tomorrow, could we explain it?