Most of the reliability work around AI agents right now goes into the model and the retrieval. We tune the prompt, we improve the ranker, we add a re-ranker on top of that. What gets far less attention is a simpler question that sits one step later in the pipeline: by the time a piece of context actually reaches the model, is it still worth acting on?
That question is what I have been building around, and this is a short, honest account of why I think it matters.
An agent acts on whatever context reaches it. If that context is stale, if it cannot be traced to a source, or if it contradicts itself, the agent usually acts on it anyway. The failure is quiet. There is no exception, no red line in a log, nothing that says the system relied on something it should not have. In a low-stakes workflow that is a nuisance. In a regulated or high-value one, that missing record is a real exposure, because afterwards there is no way to show that the context was ever checked.
Retrieval scoring does not close this gap. A ranked list tells you which chunks are most similar to the query. It does not tell you whether the top chunk is three days out of date, whether it can be attributed to anything, or whether it agrees with the chunk below it. Similarity and integrity are different questions.
I use three properties, because they are the ones that actually break in practice:
A piece of context can score high on relevance and fail all three. That is the case worth catching.
The part I care about most is not the score. It is that the result is independently checkable.
Every evaluation is written to a ledger and signed with an Ed25519 key. Anyone can take a verdict and verify it offline: no account, no API key, no call back to my server. That property is deliberate. A trust layer that you have to trust on faith is not a trust layer. If I tell you the context was checked, you should be able to confirm that claim yourself, later, without me in the loop.
This is the difference between a promise and proof. A promise is a sentence in a pitch. Proof is a signature you can verify after I am gone.
I would rather be precise than impressive, so plainly:
Built and running today: the evaluation engine that scores freshness, attribution, and coherence with a decay-adjusted model; signed, ledger-backed, offline-verifiable verdicts; an interface over the Model Context Protocol; and a hosted endpoint with a live before-and-after demo. The specification is published under an MIT licence.
Not built yet, on the roadmap: a pass / warn / refresh / block safety harness, and a control-plane dashboard with telemetry. I mention them so the picture is honest, not to imply they exist.
There is a before-and-after demo here: https://freshcontext.dev/context-integrity-demo Run something through it, then verify the verdict yourself. If you build agents, retrieval, or memory systems and this problem is one you have hit, I would value hearing how it shows up for you. That is the part I am still learning.
— Immanuel Gabriel, Founder, FreshContext