Agent memory traces are half standard: log the rest under your own conventions

When an agent gives a wrong answer, the prompt and the tool calls are usually in the trace. What it remembered, and why that memory came up, often is not. Memory reads and writes shape the next step as much as a tool result does, but they tend to be logged as opaque function calls or not logged at all. That is starting to change: the OpenTelemetry GenAI span conventions now describe a memory operation as its own span. (OpenTelemetry (semantic-conventions-genai repository))

Standards bodies have also started to say what a memory trace should contain. OWASP's Agent Observability Standard (AOS) says retrieval attributes should cover the memory type, the query and the retrieved content. That records what was asked and what came back. It does not record how good the result was, or whether it was still true. (OWASP Agent Observability Standard (AOS))

This article separates the parts of a memory trace a standard gives you from the parts you still have to define yourself. It also suggests how to keep that second set portable. (OpenTelemetry (semantic-conventions-genai repository), OWASP Agent Observability Standard (AOS))

The OpenTelemetry memory span covers two kinds of work. In the spec's words, it "describes a memory operation, including memory store lifecycle operations and memory record operations." The split is useful. Creating or deleting a store is an infrastructure event. Writing, updating or deleting a record changes what the agent will believe the next time it runs. (OpenTelemetry (semantic-conventions-genai repository))

Because memory operations are spans, they sit in the same trace as model calls and tool calls, under the same parent. You can follow a bad answer back to the retrieval that fed it without joining logs from a separate system. (OpenTelemetry (semantic-conventions-genai repository))

AOS covers the retrieval side. If you record memory type, query and retrieved content, a reviewer can tell three failures apart: the agent searched the wrong kind of memory, it phrased the query badly, or it got back something irrelevant. Each of those has a different fix. (OWASP Agent Observability Standard (AOS))

Together, the two standards describe the shape of an operation and the inputs and outputs of a retrieval. That is enough to replay what happened. It is not enough to judge it. (OpenTelemetry (semantic-conventions-genai repository), OWASP Agent Observability Standard (AOS))

The catch is that the score means whatever the vendor's scorer means. A relevance value from one system and its threshold cannot be compared directly with another's. None of the sources here defines a shared scale or a standard attribute for it. AOS asks for the retrieved content, not a judgement of its quality. (Langfuse, OWASP Agent Observability Standard (AOS))

Staleness is handled differently again. In EverOS, reflection merges memories and retires the outdated one, which then stops coming back from search. The design makes sense, but it means staleness is an event inside the memory system, not something you can read off a retrieval span. If you only log searches, a retired memory simply disappears from your traces, and you lose the record of why. (Langfuse)

Decay, meaning the downweighting of memories as they age, is not covered by the OpenTelemetry span or the AOS attributes cited here. Without a convention, a trace cannot show why an old memory was missing. It may have scored too low, it may have been retired, or it may never have been written. (OpenTelemetry (semantic-conventions-genai repository), OWASP Agent Observability Standard (AOS), Langfuse)

That puts the burden on the memory layer and on your traces. If the model will not reliably reject stale content, you need to see when stale content reached it, and whether the memory system had already marked it as outdated. (arXiv, Langfuse)

Deletion follows the same pattern. One arXiv study reports that a mutation-time hook recovers intent-aware deletion at 78-85%. Taken plainly, a mutation-time hook runs when a memory is changed, not when it is read. For observability, the location matters. A trace that only captures retrieval misses the moment the deletion decision was made. (arXiv)

So a useful memory trace records changes as carefully as searches: what changed, what was retired or deleted, and what caused it. The OpenTelemetry memory span already covers record operations, so this means filling in an existing span, not inventing a new one. (OpenTelemetry (semantic-conventions-genai repository), arXiv)

The practical answer today is a schema in two layers. The first follows the standards: one memory span per operation, store lifecycle kept separate from record operations, and for every retrieval the memory type, query and retrieved content. (OpenTelemetry (semantic-conventions-genai repository), OWASP Agent Observability Standard (AOS))

The second layer is yours, and it should be obviously yours. Put relevance, threshold outcome, record age, and retirement or merge events under your own attribute namespace. Then nobody downstream mistakes them for standard fields, and you can rename them cleanly if a convention appears. (Langfuse)

The example below uses placeholder names for both layers. Map the first layer to the attribute names in the current OpenTelemetry GenAI conventions. Those names are not reproduced here and may have changed since these sources were published. (OpenTelemetry (semantic-conventions-genai repository), OWASP Agent Observability Standard (AOS))

A memory search span with standard-shaped retrieval attributes, plus a custom namespace for relevance, age and retired records.

import json
import secrets
import time
from difflib import SequenceMatcher

NOW = time.time()
STORE = {
    "mem-a": {"type": "semantic", "text": "Customer prefers invoices by email", "created": NOW, "retired": False},
    "mem-b": {"type": "semantic", "text": "Customer prefers invoices by post", "created": NOW, "retired": True},
}

def relevance(query, text):
    return SequenceMatcher(None, query.lower(), text.lower()).ratio()

def search_memory(query):
    started = time.time()
    live = sorted(
        ((relevance(query, m["text"]), key, m) for key, m in STORE.items() if not m["retired"]),
        reverse=True,
    )
    best = next(iter(live), None)
    score, mem_id, mem = best if best else (None, None, None)
    span = {
        "name": "memory.search",
        "span_id": secrets.token_hex(8),
        "start": started,
        "end": time.time(),
        "attributes": {
            "std.memory.type": mem["type"] if mem else None,
            "std.memory.query": query,
            "std.memory.content": mem["text"] if mem else None,
            "app.memory.id": mem_id,
            "app.memory.relevance": score,
            "app.memory.age_seconds": (started - mem["created"]) if mem else None,
            "app.memory.retired_ids": sorted(k for k, m in STORE.items() if m["retired"]),
        },
    }
    return mem, span

if __name__ == "__main__":
    _, span = search_memory("how does the customer want invoices")
    print(json.dumps(span))

Tested: the example and its tests.

Staleness: log every merge and retirement as a record operation, with a reference to the memory that replaced it. You can then check whether retired memories still appear anywhere, and whether the replacement is the one being retrieved. (Langfuse, OpenTelemetry (semantic-conventions-genai repository))

Deletion: when a user asks the agent to forget something, trace the change and check it at write time. The 78-85% figure comes from one study's setup and is not a target for yours. It is a reason to put the check where the change happens. (arXiv)

The OpenTelemetry GenAI conventions and AOS are both still changing, and the versions cited here are undated. Check the current text before you hard-code attribute names. (OpenTelemetry (semantic-conventions-genai repository), OWASP Agent Observability Standard (AOS))

The two benchmark figures come from specific studies with their own setups. They show that stale memory and deletion are hard. They do not predict how your system will score. (arXiv, arXiv)

The EverOS approach is one vendor's design. It is a useful reference for what to record. It is not evidence that relevance thresholds and reflection are the right model for every memory system. (Langfuse)

On Monday, pick one agent and check whether its memory reads and writes show up as spans in the same trace as its model calls. Add the retrieval attributes AOS names, then put relevance, age and retirement events under a namespace that is clearly your own. Finally, put a few stale memories in the store on purpose and watch what the agent does with them.

Researched and drafted with AI tools; reviewed and edited by me. First published at codewithrashid.com.