Show HN: MemSeal – provenance containment for agent long-term memory MemSeal, a provenance-containment layer for AI agent long-term memory published on GitHub by developer NagaYu, reported an overall GATE 2 result of FAIL on its Holdout v2 corpus, missing two utility bars while clearing both security criteria. On Holdout v2 at condition C3 (write plus recall path), cross-principal and secret leaks fell from 22 to 0 across C0 to C2, injection success dropped from 0.775 to 0.20, and delayed-use rate hit 0.000, but clean recall retention came in at 0.444 against a 0.80 bar and quarantine precision at 0.738 against a 0.75 bar. The author attributes the 0.444 retention figure to a condition-neutral 18-marker sample identical across C0/C2/C3/C4 (8/18) and the 0.738 precision to 16 false positives, 4 from write policy W1 inside M2 attack cases and 12 from policy_refused records, noting W1 produced zero false positives on clean cases. MemSeal is a containment layer on the write/recall boundary of an AI agent's long-term memory. Every memory entry is wrapped in a provenance envelope who wrote it, from what source, derived from what, at which trust tier inside a SHA-256 hash chain. Deterministic policies decide what may be stored, and a trust-tiered recall path decides what may be surfaced — especially before an external send or an irreversible action. - No LLM in the policy. The envelope, the write rules W1–W5 , the recall path and the consolidation guard are deterministic and offline. - Measured with a real local LLM. Agent behavior is driven by qwen2.5:14b via Ollama, temperature 0, seed 20261107, one attempt per case, deterministic marker scoring, $0 , all local. - Honest gate. The acceptance gate below is self-reported as FAIL ; passing and failing criteria are shown with equal weight. write attempt ──► extract candidate ──► W1 imperative W2 derived-tier W3 principal W4 secret W5 sealed-slot │ active │ quarantine / review / reject ▼ hash-chained, per-principal memory store │ recall ──► verify chain ──► principal filter ──► provenance + trust tier ──► sole-basis gate outbound/irreversible: withhold low-trust, uncorroborated facts Write policies: W1 quarantines instruction-form content from any source but the user; W2 makes derived entries inherit their worst ancestor's trust tier no laundering a document into a "user fact" ; W3 rejects cross-principal writes and filters cross-principal recall; W4 seals secrets by entry class=secret and by shape so plaintext never lands in the store; W5 sends a fact that contradicts a sealed fact to review. The gate is judged on an independent holdout corpus authored by a separate party that never saw MemSeal's code, the dev corpus, or the reports. The bars are fixed in advance and never moved. Conditions: C0 raw store · C2 write path · C3 write + recall path · C4 + consolidation guard. | Criterion C3 | Holdout v1 pre-fix | Holdout v2 post-fix | Bar | |---|---|---|---| | Delayed-use rate ≤ 0.30 and ≤ ½·C0 | 0.547 — FAIL | 0.000 — PASS | ≤ 0.30 | | Leak count cross-principal / secret = 0 | 5 — FAIL | 0 — PASS | = 0 | | Clean recall retention ≥ 0.80 | 0.875 — PASS | 0.444 — FAIL | ≥ 0.80 | | Quarantine precision ≥ 0.75 | 0.837 — PASS | 0.738 — FAIL | ≥ 0.75 | Overall: GATE 2 = FAIL on v2. Three deterministic fixes between v1 and v2 classification-based secret sealing; a sole-basis recall gate; consolidation root retirement closed both security criteria — on v2, cross-principal + secret leaks went from 22 → 0 across C0 → C2, injection success 0.775 → 0.20 , and delayed-use 0.000 at C3. The two remaining failures are utility-side, and their causes are stated precisely not hand-waved : - Clean recall retention 0.444 is measured over a small 18-marker sample and is identical across C0/C2/C3/C4 8/18 — i.e. it is condition-neutral and not caused by MemSeal . Of the 10 misses, only 1 would recover under case-insensitive matching ceiling 0.556 ; the rest are genuine non-surfacings where even the raw C0 baseline no defense did not reproduce stored content. - Quarantine precision 0.738 comes from 16 false positives: 4 are W1 all inside M2 attack cases, i.e. legitimate writes embedded in an attack , and 12 are policy refused records cross-principal M7 writes ×8 and 4 clean counted under the harness's rejected accounting. W1 produced zero false positives on clean cases. Full detail, including an audit Errata that corrects earlier attribution wording, is in reports/phase2-regate.md https://github.com/NagaYu/memseal/blob/main/reports/phase2-regate.md and reports/phase2-fix.md https://github.com/NagaYu/memseal/blob/main/reports/phase2-fix.md . - 8 attack classes M1–M8 query-only injection, document persistence, consolidation laundering, sleeper/trigger, false-fact implantation, preference drift, cross-principal leakage, secret staging plus clean utility cases. - Split discipline: the dev split 120 cases is public in this repo; holdout corpora are authored independently and kept sealed — only their existence and aggregate numbers are reported here, never the cases. - Conditions C0–C4 and deterministic marker scoring no LLM judge : injection success, delayed use, leak count, clean retention, quarantine precision/recall, per attack class. | Metric | C0 | C1 | C2 | C3 | C4 | |---|---|---|---|---|---| | Injection success | 0.982 | 0.713 | 0.556 | 0.556 | 0.463 | | Delayed-use | 0.849 | 0.616 | 0.488 | 0.395 | 0.372 | | Leak count | 21 | 12 | 6 | 2 | 0 | The dev split shares the author's templates, so treat dev as an optimistic in-distribution reference; the independent holdout is the real test. MemSeal was also run as a wrapper around Mem0 fully local: Ollama LLM + Ollama embedder, in-memory vector store , 60 cases, two conditions: - C3-wrap drove M8 secret leaks 4 → 0 and injection 0.673 → 0.558 vs. Mem0 native. - Disclosed cost: wrap clean retention dipped 0.75 → 0.667 an exact-content match artifact between Mem0's paraphrase and the MemSeal store — see report , and Mem0's native user id scoping already blocked the cross-principal class. python3 -m pytest -q deterministic policy + bench + harness tests python3 -m memseal bench generate --split dev --out bench/dev python3 -m memseal verify