Agent memory traces are half standard: log the rest under your own conventions

A developer's analysis finds that OpenTelemetry's GenAI span conventions and OWASP's Agent Observability Standard (AOS) together cover only part of what an agent memory trace should record. OpenTelemetry now defines memory operations as their own spans, letting memory reads and writes sit in the same trace as model and tool calls, while AOS specifies retrieval attributes for memory type, query and retrieved content. Neither standard defines a shared relevance scale or a way to read staleness off a retrieval span, so the piece recommends logging those vendor-specific signals under custom conventions kept portable.

When an agent gives a wrong answer, the prompt and the tool calls are usually in the trace. What it remembered, and why that memory came up, often is not. Memory reads and writes shape the next step as much as a tool result does, but they tend to be logged as opaque function calls or not logged at all. That is starting to change: the OpenTelemetry GenAI span conventions now describe a memory operation as its own span. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md Standards bodies have also started to say what a memory trace should contain. OWASP's Agent Observability Standard AOS says retrieval attributes should cover the memory type, the query and the retrieved content. That records what was asked and what came back. It does not record how good the result was, or whether it was still true. OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ This article separates the parts of a memory trace a standard gives you from the parts you still have to define yourself. It also suggests how to keep that second set portable. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ The OpenTelemetry memory span covers two kinds of work. In the spec's words, it "describes a memory operation, including memory store lifecycle operations and memory record operations." The split is useful. Creating or deleting a store is an infrastructure event. Writing, updating or deleting a record changes what the agent will believe the next time it runs. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md Because memory operations are spans, they sit in the same trace as model calls and tool calls, under the same parent. You can follow a bad answer back to the retrieval that fed it without joining logs from a separate system. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md AOS covers the retrieval side. If you record memory type, query and retrieved content, a reviewer can tell three failures apart: the agent searched the wrong kind of memory, it phrased the query badly, or it got back something irrelevant. Each of those has a different fix. OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ Together, the two standards describe the shape of an operation and the inputs and outputs of a retrieval. That is enough to replay what happened. It is not enough to judge it. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ The catch is that the score means whatever the vendor's scorer means. A relevance value from one system and its threshold cannot be compared directly with another's. None of the sources here defines a shared scale or a standard attribute for it. AOS asks for the retrieved content, not a judgement of its quality. Langfuse https://langfuse.com/integrations/other/everos , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ Staleness is handled differently again. In EverOS, reflection merges memories and retires the outdated one, which then stops coming back from search. The design makes sense, but it means staleness is an event inside the memory system, not something you can read off a retrieval span. If you only log searches, a retired memory simply disappears from your traces, and you lose the record of why. Langfuse https://langfuse.com/integrations/other/everos Decay, meaning the downweighting of memories as they age, is not covered by the OpenTelemetry span or the AOS attributes cited here. Without a convention, a trace cannot show why an old memory was missing. It may have scored too low, it may have been retired, or it may never have been written. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ , Langfuse https://langfuse.com/integrations/other/everos That puts the burden on the memory layer and on your traces. If the model will not reliably reject stale content, you need to see when stale content reached it, and whether the memory system had already marked it as outdated. arXiv https://arxiv.org/abs/2605.06527 , Langfuse https://langfuse.com/integrations/other/everos Deletion follows the same pattern. One arXiv study reports that a mutation-time hook recovers intent-aware deletion at 78-85%. Taken plainly, a mutation-time hook runs when a memory is changed, not when it is read. For observability, the location matters. A trace that only captures retrieval misses the moment the deletion decision was made. arXiv https://arxiv.org/abs/2606.15903 So a useful memory trace records changes as carefully as searches: what changed, what was retired or deleted, and what caused it. The OpenTelemetry memory span already covers record operations, so this means filling in an existing span, not inventing a new one. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , arXiv https://arxiv.org/abs/2606.15903 The practical answer today is a schema in two layers. The first follows the standards: one memory span per operation, store lifecycle kept separate from record operations, and for every retrieval the memory type, query and retrieved content. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ The second layer is yours, and it should be obviously yours. Put relevance, threshold outcome, record age, and retirement or merge events under your own attribute namespace. Then nobody downstream mistakes them for standard fields, and you can rename them cleanly if a convention appears. Langfuse https://langfuse.com/integrations/other/everos The example below uses placeholder names for both layers. Map the first layer to the attribute names in the current OpenTelemetry GenAI conventions. Those names are not reproduced here and may have changed since these sources were published. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ A memory search span with standard-shaped retrieval attributes, plus a custom namespace for relevance, age and retired records. python import json import secrets import time from difflib import SequenceMatcher NOW = time.time STORE = { "mem-a": {"type": "semantic", "text": "Customer prefers invoices by email", "created": NOW, "retired": False}, "mem-b": {"type": "semantic", "text": "Customer prefers invoices by post", "created": NOW, "retired": True}, } def relevance query, text : return SequenceMatcher None, query.lower , text.lower .ratio def search memory query : started = time.time live = sorted relevance query, m "text" , key, m for key, m in STORE.items if not m "retired" , reverse=True, best = next iter live , None score, mem id, mem = best if best else None, None, None span = { "name": "memory.search", "span id": secrets.token hex 8 , "start": started, "end": time.time , "attributes": { Layer one: the concepts AOS asks for. Map to current OTel GenAI names. "std.memory.type": mem "type" if mem else None, "std.memory.query": query, "std.memory.content": mem "text" if mem else None, Layer two: your own conventions, clearly namespaced. "app.memory.id": mem id, "app.memory.relevance": score, "app.memory.age seconds": started - mem "created" if mem else None, "app.memory.retired ids": sorted k for k, m in STORE.items if m "retired" , }, } return mem, span if name == " main ": , span = search memory "how does the customer want invoices" print json.dumps span Tested: the example and its tests https://github.com/code-with-rashid/field-notes-code/tree/main/agent-memory-traces-half-standard . Staleness: log every merge and retirement as a record operation, with a reference to the memory that replaced it. You can then check whether retired memories still appear anywhere, and whether the replacement is the one being retrieved. Langfuse https://langfuse.com/integrations/other/everos , OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md Deletion: when a user asks the agent to forget something, trace the change and check it at write time. The 78-85% figure comes from one study's setup and is not a target for yours. It is a reason to put the check where the change happens. arXiv https://arxiv.org/abs/2606.15903 The OpenTelemetry GenAI conventions and AOS are both still changing, and the versions cited here are undated. Check the current text before you hard-code attribute names. OpenTelemetry semantic-conventions-genai repository https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md , OWASP Agent Observability Standard AOS https://aos.owasp.org/spec/trace/extend opentelemetry/ The two benchmark figures come from specific studies with their own setups. They show that stale memory and deletion are hard. They do not predict how your system will score. arXiv https://arxiv.org/abs/2606.15903 , arXiv https://arxiv.org/abs/2605.06527 The EverOS approach is one vendor's design. It is a useful reference for what to record. It is not evidence that relevance thresholds and reflection are the right model for every memory system. Langfuse https://langfuse.com/integrations/other/everos On Monday, pick one agent and check whether its memory reads and writes show up as spans in the same trace as its model calls. Add the retrieval attributes AOS names, then put relevance, age and retirement events under a namespace that is clearly your own. Finally, put a few stale memories in the store on purpose and watch what the agent does with them. Researched and drafted with AI tools; reviewed and edited by me. First published at codewithrashid.com https://codewithrashid.com/blog/agent-memory-traces-half-standard .