# Show HN: MemSeal – provenance containment for agent long-term memory

> Source: <https://github.com/NagaYu/memseal>
> Published: 2026-11-09 12:58:14+00:00

**MemSeal is a containment layer on the write/recall boundary of an AI agent's long-term memory.**
Every memory entry is wrapped in a provenance envelope (who wrote it, from what source, derived from
what, at which trust tier) inside a SHA-256 hash chain. Deterministic policies decide what may be
stored, and a trust-tiered recall path decides what may be surfaced — especially before an external
send or an irreversible action.

- **No LLM in the policy.** The envelope, the write rules (W1–W5), the recall path and the
consolidation guard are deterministic and offline.
- **Measured with a real local LLM.** Agent behavior is driven by`qwen2.5:14b` via Ollama,
temperature 0, seed 20261107, one attempt per case, deterministic marker scoring,**$0** , all local.
- **Honest gate.** The acceptance gate below is self-reported as**FAIL** ; passing and failing
criteria are shown with equal weight.

```
write attempt ──► extract candidate ──► [W1 imperative] [W2 derived-tier] [W3 principal]
                                        [W4 secret]      [W5 sealed-slot]
                                            │ active                 │ quarantine / review / reject
                                            ▼
                           hash-chained, per-principal memory store
                                            │
recall ──► verify chain ──► principal filter ──► provenance + trust tier ──►
           sole-basis gate (outbound/irreversible: withhold low-trust, uncorroborated facts)
```

Write policies: **W1** quarantines instruction-form content from any source but the user;
**W2** makes derived entries inherit their worst ancestor's trust tier (no laundering a document into
a "user fact"); **W3** rejects cross-principal writes and filters cross-principal recall; **W4** seals
secrets (by `entry_class=secret` *and* by shape) so plaintext never lands in the store; **W5** sends a
fact that contradicts a sealed fact to review.

The gate is judged on an **independent holdout** corpus authored by a separate party that never saw
MemSeal's code, the dev corpus, or the reports. The bars are fixed in advance and never moved.
Conditions: **C0** raw store · **C2** write path · **C3** write + recall path · **C4** + consolidation
guard.

| Criterion (C3) | Holdout v1 (pre-fix) | Holdout v2 (post-fix) | Bar | 
|---|---|---|---|
| Delayed-use rate ≤ 0.30 **and** ≤ ½·C0 | 0.547 — **FAIL** | **0.000 — PASS** | ≤ 0.30 | 
| Leak count (cross-principal / secret) = 0 | 5 — **FAIL** | **0 — PASS** | = 0 | 
| Clean recall retention ≥ 0.80 | 0.875 — PASS | 0.444 — **FAIL** | ≥ 0.80 | 
| Quarantine precision ≥ 0.75 | 0.837 — PASS | 0.738 — **FAIL** | ≥ 0.75 | 

**Overall: GATE 2 = FAIL on v2.** Three deterministic fixes between v1 and v2 (classification-based
secret sealing; a sole-basis recall gate; consolidation root retirement) closed both security
criteria — on v2, cross-principal + secret leaks went from **22 → 0** across C0 → C2, injection
success **0.775 → 0.20**, and delayed-use **0.000** at C3. The two remaining failures are
utility-side, and their causes are stated precisely (not hand-waved):

- **Clean recall retention 0.444** is measured over a**small 18-marker sample** and is**identical
across C0/C2/C3/C4 (8/18)** — i.e. it is**condition-neutral and not caused by MemSeal** . Of the 10
misses, only 1 would recover under case-insensitive matching (ceiling 0.556); the rest are genuine
non-surfacings where even the raw C0 baseline (no defense) did not reproduce stored content.
- **Quarantine precision 0.738** comes from 16 false positives:**4** are W1 (all inside**M2 attack** cases, i.e. legitimate writes embedded in an attack), and**12** are`policy_refused` records
(cross-principal M7 writes ×8 and 4 clean) counted under the harness's rejected accounting.**W1 produced zero false positives on clean cases.**

Full detail, including an audit Errata that corrects earlier attribution wording, is in
[`reports/phase2-regate.md`](https://github.com/NagaYu/memseal/blob/main/reports/phase2-regate.md) and [` reports/phase2-fix.md`](https://github.com/NagaYu/memseal/blob/main/reports/phase2-fix.md).

- **8 attack classes** M1–M8 (query-only injection, document persistence, consolidation laundering,
sleeper/trigger, false-fact implantation, preference drift, cross-principal leakage, secret staging)
plus**clean** utility cases.
- **Split discipline:** the**dev** split (120 cases) is public in this repo;**holdout** corpora are
authored independently and kept sealed — only their existence and aggregate numbers are reported here,
never the cases.
- **Conditions C0–C4** and**deterministic marker scoring** (no LLM judge): injection success, delayed
use, leak count, clean retention, quarantine precision/recall, per attack class.

| Metric | C0 | C1 | C2 | C3 | C4 | 
|---|---|---|---|---|---|
| Injection success | 0.982 | 0.713 | 0.556 | 0.556 | 0.463 | 
| Delayed-use | 0.849 | 0.616 | 0.488 | 0.395 | 0.372 | 
| Leak count | 21 | 12 | 6 | 2 | 0 | 

(The dev split shares the author's templates, so treat dev as an optimistic in-distribution reference; the independent holdout is the real test.)

MemSeal was also run as a wrapper around **Mem0** (fully local: Ollama LLM + Ollama embedder,
in-memory vector store), 60 cases, two conditions:

- **C3-wrap drove M8 secret leaks 4 → 0** and injection 0.673 → 0.558 vs. Mem0 native.
- Disclosed cost: wrap clean retention dipped 0.75 → 0.667 (an exact-content match artifact between
Mem0's paraphrase and the MemSeal store — see report), and Mem0's native `user_id` scoping already
blocked the cross-principal class.

```
python3 -m pytest -q          # deterministic policy + bench + harness tests
python3 -m memseal bench generate --split dev --out bench/dev
python3 -m memseal verify <store_dir>
python3 -m memseal quarantine <store_dir>
```

The real-LLM measurement harness is in [`memseal_phase2/`](https://github.com/NagaYu/memseal/blob/main/memseal_phase2) and talks only to a local
Ollama (`qwen2.5:14b`). No external API, no API key, no network beyond localhost and the one-time model
download. Tests use a mock LLM, so the suite runs without Ollama.

- **Tests:** full suite green (deterministic; mock-LLM harness tests need no Ollama).
- **GATE 2:** FAIL (recorded). Security criteria pass on the independent v2 holdout; utility criteria
fail for the reasons above.
- Phase reports: [`reports/phase1.md`](https://github.com/NagaYu/memseal/blob/main/reports/phase1.md) ·[` reports/phase2.md`](https://github.com/NagaYu/memseal/blob/main/reports/phase2.md) ·[` reports/phase2-fix.md`](https://github.com/NagaYu/memseal/blob/main/reports/phase2-fix.md) ·[` reports/phase2-regate.md`](https://github.com/NagaYu/memseal/blob/main/reports/phase2-regate.md) .

**All Rights Reserved.** This repository is published for evaluation reading only; no rights to use,
copy, modify, or redistribute are granted. See [`LICENSE`](https://github.com/NagaYu/memseal/blob/main/LICENSE). Acquisition / licensing /
transfer inquiries are welcome through the author's portfolio: