You Recorded Every Event. Can You Still Reconstruct the Execution?

A developer working on AI monetization infrastructure argues that recording every runtime event in a distributed AI workflow is insufficient to reconstruct an execution, because the relationships between events — retries, parallel branches, fallbacks, and asynchronous continuations — are lost even when individual events are preserved. The engineer contends that request_id and HTTP request/response boundaries break down once work outlives the original request, and calls for preserving identity and lineage rather than events alone.

Imagine that, months after an AI workflow ran, you want to reconstruct exactly what happened. The customer asked for a research report. The report was eventually delivered, and your system recorded every relevant runtime event along the way. Nothing is obviously missing. You have events for planning, search, tool calls, model calls, validation and the final successful result. Each record has a timestamp. The provider calls are there. The failures are there too. Your data might look something like this: 10:00:01 request accepted 10:00:03 planning started 10:00:07 search 10:00:08 search 10:00:12 tool call 10:00:15 tool call failed 10:00:18 tool call 10:00:26 model call 10:00:31 validation failed 10:00:37 model call 10:00:44 validation succeeded 10:00:45 report delivered At first glance, this looks like a pretty good execution history. But try answering a few questions from it. Was the second tool call a retry of the failed one, or a different operation? Did the two searches run sequentially or as parallel branches? Was the second model call a retry, a fallback, or a new stage of the workflow? And if some of this work continued asynchronously, did it still belong to the execution initiated by the original request? The individual events may all be accurate. The structure that connected them may already be gone. That's the part I've been thinking about while working on AI monetization infrastructure. It's tempting to treat reconstruction as an aggregation problem: preserve the events now, group them later, and the execution history will still be there when you need it. I'm becoming less convinced that this is enough. A distributed workflow is not only a collection of things that happened. It also contains relationships: one operation spawned another, two operations ran as siblings, an attempt retried an earlier attempt, a worker continued work after the original request ended, or several branches eventually contributed to the same result. If those relationships disappear, having every event does not necessarily give us the execution back. We didn't lose the events. We lost the relationships between them. That suggests a different engineering question. Not only: What events should an AI runtime preserve? But: What identity and lineage must survive if we want to reconstruct how those events belonged together? HTTP gives us a very convenient mental model: request ↓ work ↓ response For simple synchronous operations, that model can also provide a useful boundary for observability. A request arrives, the application performs some work, returns a response, and much of what we care about happens within that lifetime. Distributed AI workflows can break that assumption very quickly. Suppose generating the research report takes long enough that we don't want the client to keep an HTTP connection open. The API accepts the request, creates some work and returns 202 Accepted . HTTP request ↓ accepted ↓ enqueue job ↓ 202 Accepted X │ │ execution continues ↓ worker ↓ planning ↓ fan-out / | \ / | \ search search tool A B C ↓ retry \ | / \ | / aggregation ↓ validation ↓ outcome The request may have lived for a few hundred milliseconds. The work it initiated may live for minutes. That difference matters because a request id can still correctly identify the interaction that entered the system without necessarily being the right identity for everything that happens afterward. The queue message may be delivered later. A worker may create several child jobs. One branch may retry independently. Another may call an external service and wait for a callback. The workflow may pause and resume after the original process that handled the HTTP request no longer exists. We can propagate the original request context through those boundaries, and doing so is extremely useful. Distributed tracing is specifically designed to carry context across services and process boundaries so related operations can remain observable as part of a distributed flow. But propagation does not make the original request and the resulting domain execution the same concept. Consider: req 123 ↓ execution 123 │ ├── search job A ├── search job B └── tool job C ↓ FAILED ↓ retry The request tells us where this interaction entered the system. What we're trying to reconstruct later is something slightly different: the logical work that continued because of it. That distinction becomes more important when work can resume without a new customer request, when one request initiates multiple independent executions, or when a later callback continues an execution that started somewhere else. So I don't think the useful conclusion is simply: Replace request IDs with execution IDs. We still want request identity. It answers a real operational question. The problem is assuming that one identifier can represent every kind of identity we care about. A request belongs to the transport interaction. An execution may need to survive beyond that interaction. And once an execution can survive its request, another problem appears almost immediately. What happens when the same logical work is attempted more than once? Retries make the identity problem harder because one piece of logical work can produce multiple physical attempts. Suppose one branch of our research workflow calls an external tool: execution 123 ↓ tool call ↓ attempt 01 ↓ provider ↓ timeout From our side, the call timed out. We don't know whether the provider rejected it, started processing it, completed it but failed to return the response, or consumed resources before something else went wrong. So we retry: execution 123 ↓ tool call │ ├── attempt 01 │ ↓ │ timeout │ └── attempt 02 ↓ success From the workflow's perspective, this may still be one logical operation: call the tool and obtain a result. From the runtime's perspective, two attempts happened. That difference matters because several questions that look similar are actually independent. Did the retry produce duplicated application state? An idempotency mechanism may help us prevent or detect that. Did the provider execute the first attempt despite our timeout? That depends on evidence we may or may not have. Did both attempts consume billable resources? That's another question. And if both consumed resources, should both eventually be associated with the execution that produced the customer outcome? That's an attribution question we haven't answered yet. Idempotency is particularly easy to overextend conceptually. An idempotency key can help a system recognize repeated processing of the same logical operation and avoid applying the same effect more than intended. But that does not mean only one physical attempt occurred. A retry can be safe from the perspective of application state while still representing additional runtime work. For reconstruction, preserving only the final logical state can therefore hide something important: logical operation ↓ SUCCESS Compare that with: logical operation │ ├── attempt 01 │ ↓ │ timeout │ └── attempt 02 ↓ SUCCESS Both representations may describe the same final application state. They do not preserve the same execution history. This is why I find it useful to distinguish, at least conceptually, between an execution identity and an attempt identity . The execution identifies the logical work we're trying to follow. The attempt distinguishes a particular try at performing some part of that work. I don't think every system needs those exact names or separate persisted identifiers for every operation. The useful distinction is semantic, not terminological. If multiple attempts can happen and those attempts matter to questions we may ask later, the runtime needs some way to preserve that relationship. Otherwise: tool call timeout tool call success leaves us trying to infer whether we observed a retry, two independent operations, or something else entirely. And retries are still the easy shape. Once one execution starts creating several pieces of work in parallel, a flat sequence of individually accurate events becomes even less representative of what actually happened. Fan-out makes the problem more obvious. Our research workflow might reach a planning stage and perform several operations in parallel: execution 123 ↓ planning ↓ fan-out / | \ / | \ search A search B tool C \ | / \ | / aggregation ↓ validation ↓ outcome Each branch can produce perfectly accurate events. Stored individually, the data could look something like this: 10:00:07 search success 10:00:08 search success 10:00:09 tool call started 10:00:15 tool call failed 10:00:18 tool call started 10:00:24 tool call success 10:00:26 model call success 10:00:31 validation failed 10:00:37 model call success 10:00:44 validation success There is nothing necessarily wrong with those records. They may be exactly what happened. But a flat list does not necessarily preserve why those events exist in relation to one another. Was the tool call at 10:00:18 a retry of the failed call, or another branch? Did both searches belong to the same fan-out? Did the model call at 10:00:37 retry the earlier model call, replace it through a fallback path, or execute because validation created a new stage of work? Timestamps can help us infer some of this. Operation names, logs, traces and application metadata may provide additional clues. But inference from proximity is different from preserving the relationship itself. The same set of events can represent different execution structures: A B C D E could have happened as: A ↓ B ↓ C ↓ D ↓ E or: A ├── B │ └── D └── C └── E or even: A ↓ B ↓ C FAILED ↓ C RETRY ↓ E It is tempting to treat relationships such as parent/child, retry-of or belongs-to-execution as metadata around the real evidence. But for reconstruction, those relationships may themselves be part of the evidence. A list can tell us what exists. A lineage graph can preserve how those things belonged together. That does not mean every runtime needs to persist an elaborate execution graph. It means that if we expect to answer structural questions later, the structure cannot always be recovered from flat measurements alone. The design question therefore isn't simply how many events we retain. It's which relationships would become impossible — or dangerously ambiguous — to recover if we didn't preserve them when the execution happened. And this is where the problem starts to overlap with distributed tracing. A trace already preserves relationships between operations across a distributed system. So if we have tracing, do we actually need another notion of execution lineage at all? At this point, there is an obvious objection. A distributed trace already exists to connect work across services. If a request moves from an API to a worker, then to a model provider and an external tool, trace context can be propagated across those boundaries so that related operations remain observable as part of a distributed flow. That's exactly what tracing is good at, and I don't think it makes sense to invent a parallel model for information that tracing already preserves well. A simplified trace might give us something like: trace │ ├── API request │ └── enqueue job │ └── worker │ ├── search ├── model call └── tool call That is already much richer than a flat list of events. We can inspect timing and reconstruct important technical relationships between operations. With correctly propagated context, those relationships can survive process and service boundaries too. But there is a subtle distinction between reconstructing technical relationships and identifying the logical unit of work our domain cares about. Consider a workflow that pauses after the initial trace and resumes later because an external system sends a callback: trace 01 │ ├── request ├── planning └── external tool request time passes trace 02 │ ├── callback received ├── workflow resumed ├── validation └── outcome Depending on how the system is instrumented, representing those activities as separate traces may be completely reasonable. From the perspective of our domain, however, they may still be two parts of the same logical execution. The opposite shape is possible too. One technical operation may process a batch containing work associated with several logical executions. Fan-out, messaging and asynchronous processing can create relationships that are not always represented cleanly by assuming one trace is equivalent to one domain execution. OpenTelemetry accounts for some non-tree relationships through span links. A span can link to other span contexts without making them its parent, which is useful in cases such as asynchronous processing, batching and scatter/gather patterns. That reinforces the point rather than weakening it: distributed execution does not always fit into one simple request-shaped hierarchy. So I would be careful with an assumption like: trace id = execution id Sometimes that mapping may be useful. It is not a semantic guarantee we get from tracing itself. A trace answers an observability question about technical operations and their relationships. The application may still have a domain question about which logical execution those operations participated in. And neither one automatically answers the economic question. Suppose our trace shows that a failed tool call was followed by a retry and that both occurred before the final report was delivered. We have learned something important about the technical execution. We still haven't decided whether the cost of both attempts should be attributed to that report, whether some of the work was shared with another outcome, or which economic boundary the business wants to analyze. That leaves us with three related but different models: TECHNICAL CAUSALITY What happened across the distributed system? ↓ DOMAIN IDENTITY / LINEAGE What logical work belonged together? ↓ ECONOMIC ATTRIBUTION Which economic unit should bear that work? Information can flow between these layers. Technical traces can provide strong evidence for reconstructing domain lineage, and domain lineage can provide evidence for later economic analysis. But one layer does not automatically define the semantics of the next. The lesson isn't that tracing is insufficient. The more useful question is whether the semantics already captured by our tracing system are the same semantics we'll need when we reconstruct the execution later. If they aren't, adding more telemetry isn't necessarily the answer. We first need to decide which identities and relationships are worth making durable. Once we accept that reconstruction depends on relationships as well as events, it's easy to move too far in the other direction. We could attach an identifier to everything, persist every transition and build a detailed graph of the entire runtime. That would preserve information. It doesn't mean all of that information would be useful. A better starting point, I think, is not: What fields might we want someday? What questions would become impossible to answer if this relationship disappeared? Suppose we want to know whether two provider calls were independent operations or two attempts at the same logical work. Then we need enough information to distinguish the operation from its attempts: execution 123 │ └── tool operation │ ├── attempt 01 → timeout └── attempt 02 → success The exact representation is less important than preserving the fact that attempt 02 exists in relation to attempt 01 , rather than merely happening a few seconds later. If we want to know whether several operations were created by the same execution, some notion of execution identity and parent/child relationship becomes useful: execution 123 │ ├── search A ├── search B └── tool C And if the domain eventually produces a meaningful result that we care about independently from the execution itself, an outcome reference may be useful too: execution 123 │ ├── attempts ├── child executions └── runtime operations │ ↓ outcome 789 But outcome id is a good example of why I wouldn't turn this into a universal schema. Not every execution produces one identifiable outcome. One execution might produce several results. Several executions might contribute to one result. Some workflows may fail without producing an outcome at all. The identity model has to reflect the questions the domain actually needs to answer. A minimal conceptual event might therefore look something like this: { "event id": "evt 42", "execution id": "exec 123", "attempt id": "attempt 02", "operation": "tool call", "occurred at": "2026-09-16T08:42:17Z", "status": "failed" } This is not a schema recommendation. A real implementation might represent these relationships very differently. What matters is what each piece of information buys us. event id gives the event a stable identity. execution id gives us a logical context in which to interpret it. attempt id prevents repeated physical work from collapsing into one final logical state. occurred at tells us when the event happened rather than relying only on when we happened to receive or persist it. The useful design principle is not to maximize metadata. It's to preserve the smallest set of identities and relationships that keeps the questions we care about answerable. And in a distributed system, even preserving that structure doesn't mean the evidence will arrive neatly. Imagine two branches running in parallel: execution 123 │ ├── search A │ ↓ │ completed at 10:00:12 │ └── tool B ↓ completed at 10:00:10 If the search event reaches our evidence store immediately while the tool event is delayed by a queue or network boundary, we might persist them in the opposite order: received 10:00:12 → search A completed received 10:00:15 → tool B completed Nothing is necessarily wrong. The order in which we learned about the events is simply different from the order in which they occurred. Retries and duplicated delivery complicate this further. The same event may be delivered more than once, while evidence about an earlier attempt may arrive after evidence about the retry that followed it. Reconstruction therefore shouldn't assume that ingestion order is execution order. At minimum, it helps to preserve a stable identity for an event and distinguish when the event occurred from when the system received or persisted it: event occurred ↓ 10:00:10 │ │ network / queue delay ↓ event recorded ↓ 10:00:15 Those timestamps answer different questions. This isn't an argument for storing every possible timestamp or building a full event-sourcing architecture. The narrower point is that lineage should not depend entirely on arrival order. If an attempt is explicitly related to the operation it retried, or a child execution preserves its relationship to a parent, that structure can remain meaningful even when the evidence arrives late or out of order. Otherwise, reconstruction can quietly become an exercise in guessing relationships from timestamps. And that difference becomes much more important when the question we're asking is no longer only operational. It becomes economic. So far, none of this requires an economic use case. Execution identity, retries, causal relationships and distributed tracing are established backend concerns. AI didn't invent them. The economic relevance appears when different execution paths can consume different resources. Suppose the cost associated with our research workflow increases between two periods: August Research workflow 1,000 completed outcomes provider spend: $200 September Research workflow 1,000 completed outcomes provider spend: $310 We know more was spent. But that doesn't tell us what changed inside the execution. Maybe the workflow started retrying an external tool more often. Maybe a validation failure caused additional model calls. Maybe one branch began falling back to a more expensive model. Maybe the execution path didn't change at all and the applicable provider rate changed instead. Those are different explanations. Some require historical pricing context. Others require historical execution context. I explored the first problem in Your AI Cost Calculation Can Be Correct — and Still Be Historically Wrong https://dev.to/thelastciroandrea/your-ai-cost-calculation-can-be-correct-and-still-be-historically-wrong-4eof . Here I'm interested in the second: whether enough of the execution structure survived to explain what changed. If the question is whether retries increased, knowing that ten thousand provider calls occurred is useful but not sufficient. We also need some way to distinguish independent operations from repeated attempts at the same logical work. If the question is whether a fallback path became more common, we need to know which operations belonged to that path. If one child execution started consuming more resources, we need to be able to identify that child execution across the evidence we preserved. The economic question therefore exposes something about the earlier architecture. A system may have retained enough data to calculate total spend while retaining too little structure to explain how that spend emerged from runtime behavior. Conceptually: PROVIDER EVENTS ↓ aggregate ↓ total spend can answer a different class of questions from: execution 123 │ ├── attempt 01 │ └── tool call │ ├── attempt 02 │ └── tool call │ └── child execution └── model call ↓ reconstruct runtime behavior The first view is useful. The second preserves a different kind of information. This connects to a broader problem we've been exploring in the Licenzy Guide One AI Outcome. Many Runtime Operations. What Actually Belongs to Its Cost? https://licenzy.app/guides/one-ai-outcome-many-runtime-operations-cost . That Guide asks which runtime work should be considered when reasoning about the economics of an outcome. The engineering question I'm interested in here comes one step earlier: Did the runtime preserve enough identity and lineage to reconstruct that work in the first place? Without meaningful lineage, a later analysis may need to infer relationships from timestamps, operation names, customer references and whatever logs happen to remain. With meaningful lineage, more of those relationships can be investigated from structure that was preserved when the execution happened. That still doesn't guarantee a complete or correct economic explanation. Provider evidence can be missing. Shared work can complicate the boundary. Historical rates may need to be reconstructed separately. Evidence can arrive late or be corrected. But there is an important difference between asking the system to recover a relationship it preserved and asking an analyst to infer a relationship that disappeared. For some economic questions, execution lineage can move us from: "These events happened around the same time." toward: "These events were related as part of this execution." That's stronger evidence. It is still not the final answer. Because even if we reconstruct the execution perfectly, we haven't yet decided what that execution means economically. Imagine that we solved the reconstruction problem perfectly. Months later, we can recover the execution graph: execution 123 │ ├── search A ├── search B │ ├── tool call │ │ │ ├── attempt 01 → timeout │ └── attempt 02 → success │ ├── model call ├── validation → failed ├── fallback model call └── validation → success ↓ outcome 789 We know which attempts belonged to the same logical operation. We know which child work belonged to the execution. We know the retry relationship, the fallback path and the result eventually produced. That's a much stronger foundation for historical investigation. But it still doesn't tell us how every piece of work should be interpreted economically. Take the failed tool attempt. Technically, it belongs to the execution. It happened because the workflow was trying to produce outcome 789 . Should its cost therefore be attributed entirely to that outcome? Maybe. Now imagine a retrieval operation populated a cache that was reused by ten later executions. Technically, we may know exactly which execution created the cached result and which executions later consumed it. Which one should bear the cost? Or imagine a batch operation processed work for several customer outcomes at once. The technical relationship can be perfectly reconstructable while the economic allocation still requires a policy. This is where I think three questions need to remain separate: TRACE What technically happened? ↓ LINEAGE What work belonged together? ↓ ATTRIBUTION What economic unit should bear that work? Each layer can provide evidence for the next. None automatically defines it. Economic attribution introduces semantics that may not exist in the technical graph. Otherwise, it's easy to make a subtle leap: "We know this operation belonged to the execution." therefore: "We know where its cost belongs." Those are not necessarily equivalent statements. I started this investigation with a fairly narrow question: what identity would an AI runtime need if we wanted to reconstruct an execution later? I now think identity is only part of the answer. The events are evidence. The relationships between them can be evidence too. This isn't a new distributed-systems problem created by AI. What makes it especially interesting in AI monetization infrastructure is that different execution paths can also have different economic consequences. If the structure disappears, we may still know how much was consumed while losing part of our ability to explain how that consumption emerged from the execution. Preserving lineage gives us a better foundation for that investigation. It doesn't decide the economics for us. And that leaves me with the question I think comes next: If technical lineage tells us what happened together, who decides what should count together economically?