A follow-up to How Old Is My Data?
In a classic dashboard, stale data is a human problem: someone looks at a number that's six hours old and makes a slightly worse decision. Annoying, rarely catastrophic.
Now hand that same data to a retrieval-augmented-generation (RAG) pipeline, or to an autonomous agent. The stakes change. The system doesn't to sanity-check the timestamp — it acts. And when the data it acts on is stale, three things are true at once:
That's the worst combination in observability: a real failure that is completely invisible to the signals we currently emit.
RAG: index vs. corpus. Your vector index was built from a corpus at some point in time. The corpus keeps changing — documents get added, edited, retracted. If the re-embedding job stalls or falls behind, the index quietly drifts out of date. The retriever still returns plausible chunks; the model still writes a fluent answer. It's just answering from a version of reality that no longer exists. The quantity you care about is the age of the index relative to its source — not the age of either one alone.
Feature stores: online–offline skew. The features your model trained on and the features it serves on are supposed to match. When the online store lags the offline pipeline, predictions degrade in a way that looks like model drift but is actually data staleness wearing a costume.
Agents: stale shared state. Multi-agent systems coordinate through shared memory, scratchpads, and context. An agent reasoning over state that another agent updated ten steps ago — but which never propagated — makes locally reasonable, globally wrong decisions. This isn't a new or exotic problem: it's exactly the regime that Age of Information theory was invented to analyze in real-time control and networked systems, where an actuator acting on a stale sensor reading is a known hazard.
None of this is a new metric. Freshness — the time since the most recent update was generated at its source — has a precise name in networked-systems research: the Age of Information:
age = now − event_time(freshest record)
If a source stops producing, its age
climbs without bound while every other signal stays healthy. Data-observability vendors already compute this internally. dbt source freshness
computes it for dbt. Kafka-lag tools compute it for Kafka.
What's missing is a standard, portable way to emit it — one vocabulary that works whether the "source" is a Postgres table, a Kafka topic, a feature store, or a vector index, and that lands in any backend unchanged. That gap was tolerable when humans were the only consumers. It stops being tolerable when the consumer is a model that acts without a second look.
data.staleness.*
I've been working on a small, vendor-neutral semantic convention for data staleness, built on OpenTelemetry, plus a working reference implementation. To be clear about status: this is an independent, community project — not an official OpenTelemetry project, and not endorsed by or affiliated with OpenTelemetry or the CNCF. The convention is a proposal at "Development" stability, under discussion with the Semantic Conventions SIG (issue #3909). Names may change.
The shape is deliberately compact:
| Metric | Meaning |
|---|---|
data.staleness.age |
|
now − event_time of the freshest record (the primary signal) |
|
data.staleness.lag |
|
processing_time − event_time of the most recent record |
|
data.staleness.last_update.timestamp |
|
| Unix time of the last successful update | |
data.staleness.records.behind |
|
| positional backlog (e.g. Kafka consumer lag) | |
data.staleness.sla.* |
|
| configured threshold + breach evaluation |
The key insight for AI systems is the differential case: the age of an index relative to the corpus it was built from, or a replica relative to its primary. A single number ("the index is 40 minutes behind its source") is exactly the alert a RAG pipeline needs and rarely has.
One principle runs through the whole implementation: an unmeasurable freshness value must be visible, never faked. An empty table, a NULL MAX()
, a timeout, an empty partition — each surfaces as data.staleness.probe.errors
with a specific error.type
, not as a fabricated "0 seconds old."
This matters doubly for AI. A freshness check that silently reports "fresh" when it actually failed is worse than no check at all — because it will reassure you precisely when your RAG index or agent state is most broken.
Python SDK — point a differential probe at your index and its source:
pip install data-staleness-otel
python
from otel_staleness import StalenessMonitor, conventions as sc
from otel_staleness.probes import SQLFreshnessProbe
monitor = StalenessMonitor()
monitor.add_probe(SQLFreshnessProbe(
fetch_max_epoch=lambda: latest_source_doc_ts(),
source_name="kb_corpus", system=sc.System.POSTGRESQL,
sla_threshold_seconds=1800))
monitor.start()
Zero-code Collector — scrape SQL, Kafka, Kinesis, files, HTTP, and schema registries from YAML alone, no application code, and emit the same data.staleness.*
metrics into the same backend.
If "how stale is the data my model is answering from?" is a question you've had to answer with a bespoke query and a fragile dashboard, I'd value your input:
The convention proposal is open in the OpenTelemetry Semantic Conventions repo (#3909) — thumbs-up, comments, and concrete AI use cases genuinely help gauge demand.
Code, spec, model, and validation: https://github.com/anirudhrajreliability/otel-data-staleness (Apache-2.0).
Try it (pip install data-staleness-otel
) and tell me where it breaks.
We standardized latency and errors because machines needed portable signals to reason about. AI systems reasoning over data need a portable signal for freshness for exactly the same reason. Let's give them one.