Memory Architecture Quality Standard for LLM Assistants

A developer published version 1.1 of a memory architecture quality standard for LLM-based assistants and agents, providing an audit checklist that covers storage mapping, input validation, and write idempotency. The standard emphasizes closure in the memory loop and requires evidence for each checklist item, with criticality markers to identify blockers.

Memory Architecture Quality Standard for LLM Assistants Format: Audit checklist — place in front of you and verify item by item. Version: 1.1 2026-08-21 Scope: Any LLM-based assistants and agents with long-term memory dialog, episodic, semantic, vector, graph, multimodal , regardless of stack and platform. 1. Reference Loop Model The audit follows a generalized memory loop. Each checklist item maps to a node or edge in this model. The critical property is closure: anything that enters memory returns to the model and can reproduce itself. Most severe memory failures are edge failures, not node failures. 2. How to Conduct the Audit - Start with Section 0 map . Without a complete storage map, results from other sections are unreliable. - Work through sections A–K in order. Mark each item: yes / partial / no / n/a . - For each no and partial , record evidence : file and line, SQL query with result, dump, prompt snapshot, log entry. An assertion without evidence is not considered verified. - Criticality markers: - CRIT — direct risk of loss, poisoning, leakage, or uncontrolled memory growth. Failure = blocker. - IMP — risk of silent quality and predictability degradation. - REC — maturity and maintainability. - The verdict section "Audit Result Protocol" is issued only after completing the full map. Section 0. System Map Preparation - 0.1 A complete storage map is compiled: relational databases, file and vector indexes, JSON stores, caches RAM and disk , external data sources. CRIT - 0.2 For each store, the modules that write to it and read from it are identified read/write ownership . CRIT - 0.3 All access keys are identified: which fields are used for writing and which for retrieval; confirmed that write keys match read keys. CRIT - 0.4 All channels where model output responses, reports, action results returns to memory input are identified. CRIT - 0.5 Verified: the analyzed code matches the executed code imports checked, active implementation confirmed, no "dead" parallel version exists . CRIT - 0.6 Baseline metrics captured: storage volumes records/bytes , typical assembled context size in tokens, retrieval time. IMP - 0.7 All points where user or external content enters privileged system parts of the prompt are identified. CRIT Section A. Input Validation INGESTION - A1 All external sources files from watched directories, web content, images, output from other agents pass a trust filter before being written to memory. CRIT - A2 Secrets keys, tokens, passwords, personal data are detected and excluded before indexing and vectorization. CRIT - A3 Service content logs, diagnostic reports, service markers, prompts is flagged and excluded from context assembly while it may remain in history . CRIT - A4 Meaning extraction does not conflate form with meaning: code fragments, syntax, paths, and technical markers do not become "concepts" or "memories." IMP - A5 For each record, provenance is captured: source, time, authoring agent, session. IMP - A6 Records from untrusted sources receive a lowered weight or are placed in a separate trust zone. IMP - A7 Content embedded in multimodal inputs text in images, file metadata passes the same validation as explicit text. CRIT Section B. Write Idempotency and Integrity WRITE - B1 Writing is idempotent: repeated delivery of an event retry, update duplication, double invocation does not create a duplicate — deduplication at application level or UNIQUE constraint in schema. CRIT - B2 Deduplication direction is correct: the current record is preserved, the old duplicate is suppressed not the reverse . CRIT - B3 Deduplication is by semantic/content key, not only by "content + source" pair — cross-source duplicates are caught. IMP - B4 Writing to multiple stores message + vector + index is transactional or compensable: partial writes "message saved, vector not" are excluded. IMP - B5 Write failures are not silent: they are logged, metricked, retried with backoff; a silent return False without observability is unacceptable. CRIT - B6 Compressed representation summary and original are written consistently; an empty/failed compression does not replace the original. CRIT Section C. Growth Management GROWTH - C1 For each store, limits or a retention policy are defined maximum volume, history depth, overflow behavior . IMP - C2 Re-indexing/rebuild is idempotent: a repeated run does not multiply records. CRIT - C3 Quotas are defined: single record size, records per session/source, total volume per container. IMP - C4 Vector index is synchronized with source: deletion/suppression of a record is reflected in the index; operation order clean source → rebuild index is defined and enforced. CRIT - C5 Caches RAM structures, summarization caches have TTL or an invalidation mechanism and are included in the growth map. IMP - C6 Volume dynamics are monitored; anomalies order-of-magnitude growth in a single operation trigger alerts. IMP - C7 Multimodal input deduplication by hash does not create unbounded counter growth and does not allow "flooding" memory freshness with a stream of unique variants. IMP Section D. Retrieval Ranking and Gating RETRIEVAL - D1 Static "importance" of a record does not compensate for low relevance: context-adaptive scoring is applied importance weight is adjusted by proximity to query and a soft relevance threshold is used, below which a record does not enter the context even with high importance. CRIT - D2 Heuristic gates "conversational query → minimal memory" do not disable long-term memory entirely for legitimate queries; gate conditions are narrow and auditable. CRIT - D3 Importance heuristics are resistant to inflation: users cannot raise a record's rank by flooding, question marks, key terms, or other transparent techniques. IMP - D4 Recency boost is bounded by thresholds: fresh irrelevant content does not displace old relevant content. IMP - D5 Retrieval is deterministic across restarts: keys are persistent; non-deterministic hashes/identifiers are not used as access keys. CRIT - D6 Thresholds, limits, and retrieval weights are externalized to configuration, not hardcoded as magic numbers. REC - D7 Externally driven importance suppression archival by external content, marking a session as "resolved" by user reply is bounded by thresholds, freshness protection, and current scope of action. CRIT - D8 The embedding model matches the language s of the data; when the model is changed, re-indexing of all stores using embeddings is mandatory. CRIT Section E. Context Assembly ASSEMBLY - E1 Blind truncation is prohibited: any shortening is semantic summarization; mid-sentence fragments do not enter the prompt. CRIT - E2 The full original is stored separately from the compressed representation two-phase storage: original in history, compression in context . CRIT - E3 The compressed representation cache is invalidated on change, restoration, or re-evaluation of sources; TTL is defined. IMP - E4 Structured blocks code, tables are excluded from compression or preserved in full. IMP - E5 Budgets are defined for each injected block characters/tokens , including tool results and memories. IMP - E6 User input does not enter privileged prompt blocks priorities, system sections, memory headers without strict validation of all fields. CRIT - E7 Trusted markers system prefixes, tool markers, memory source labels cannot be imitated by user text — input is filtered for their formats. CRIT - E8 Wildcard characters % , and metacharacters in user queries are escaped in LIKE/regex memory searches. IMP - E9 Commands/actions extracted from model response text file operations, search, memory calls are validated: paths, permissions, confirmations, limits. CRIT - E10 A mechanism exists for protecting emotionally significant records from summarization and decay protected/alive memory ; the maximum number of protected records is capped. REC - E11 Critical information is placed at the beginning and end of the context window, not in the middle; for long contexts 16K+ tokens , the positional attention drop effect is accounted for ref: Liu et al., "Lost in the Middle," 2023 . IMP Section F. Feedback Loop FEEDBACK LOOP - F1 Model output responses, reports, action results is filtered before returning to memory: meta-content "analysis of analysis," service summaries, reports about the context itself is not written as regular content. CRIT - F2 The summarizer/compressor does not close the loop through the main LLM client: the call is bare no history, no saving to memory . CRIT - F3 Summarization of a summary is impossible repeated compression is a no-op . IMP - F4 A recursive artifact detector exists: meta-response signatures, control of the proportion of model-generated content in memory, alert on growth. IMP - F5 Memory recall results inserted into response text do not "leak" back into long-term memory without a filter. CRIT - F6 The path "input error → distorted response → writing distortion to memory" is broken: context assembly errors are not masked as valid content. CRIT Section G. Isolation and Trust Boundaries ISOLATION - G1 Isolation boundaries are not degenerate: a filter that in practice always passes the same value a single scope identifier is not a boundary but an imitation of one. CRIT - G2 Cross-scope transfer is possible only through an explicit controlled channel bridge, mapping, permitted associations , not through shared search. IMP - G3 Fallback retrieval branches do not return irrelevant content "just to return something": an empty result is more honest than random filling. IMP - G4 External stores belonging to other systems are connected by contract: changes to the external side's schema/semantics are detected, not silently absorbed. IMP - G5 First-write description latch on hash-based deduplication is not irrevocable: description updates and re-description on re-perception are supported. IMP - G6 Domain/scope classification is robust to rephrasing; a classification error does not mean irrecoverable loss of content visibility a recovery path exists . IMP - G7 Long-term priorities are protected from suppression by temporary boosts: maximum boost lifetime and restoration of base weights are defined. IMP Section H. Concurrency and Migrations CONCURRENCY - H1 Storage access is unified connection pool / single gateway ; multiple independent connections with mutual locks are absent. IMP - H2 Shared mutable state priorities, boosts, caches, session flags is protected from races in async/multithreaded processing. CRIT - H3 Time intervals decay, freshness, TTL are calculated from a monotonic source; system clock changes do not break the logic. IMP - H4 Schema migrations are idempotent existence check before change , accompanied by backup and dry-run; partial application is excluded or detectable. CRIT - H5 Tests are isolated from production stores: temporary/in-memory databases, mocked external APIs, no concurrent access to live data. CRIT - H6 Repeated processing of a single event idempotency key at system input level excludes double generation and double writing. IMP Section I. Observability and Recovery OBSERVABILITY & RECOVERY - I1 Memory contamination monitoring is operational: pattern signatures, quality score, alert threshold. IMP - I2 Silent degradations are excluded: every graceful fallback is accompanied by a metric/alert, not just a log line; mass replacement of memory with stubs is detectable. CRIT - I3 Backups are regular; the restore procedure has been verified in practice restore drill , not just by the existence of copies. IMP - I4 Data deletion is soft: weight suppression/archival instead of DELETE; a recovery path unarchive is implemented and tested. IMP - I5 A health check exists: ready-made commands for liveness verification, counter consistency between stores, and availability of external dependencies. IMP - I6 Availability of auxiliary infrastructure web consoles, admin panels is protected by authentication; they do not expose memory externally. CRIT - I7 A diagnostic dialog protocol exists — structured questioning of the agent about its context state "what do you see?", "what interferes?", "what's missing?" as a complement to external metrics. REC - I8 Adaptive behavioral parameters personality traits, style calibrations are persistent across restarts; restoration is verified at each startup. IMP Section J. Change Management CHANGE MANAGEMENT - J1 One logical change per observation interval; simultaneous independent modifications are excluded otherwise the cause of regression is non-localizable . CRIT - J2 The success criterion of a change is measurable: baseline captured before in tokens/numbers, not "by eye" , measurement after. IMP - J3 Every resolved incident is closed with a regression guard test; a fix without a test is considered incomplete. IMP - J4 Local validation of context assembly a snapshot of "what the model will actually see" precedes live testing. IMP - J5 Changes are reversible: one change per commit, rollback of an individual step does not pull adjacent ones. IMP - J6 A documentation cycle is maintained: diagnosis → spec → review → session log → report → post-solution pattern generalization with applicability signals ; patterns are checked for applicability at the start of each new spec. REC Section K. Quick Diagnostics Problem Smells Symptom → probable defect class. Used for quick navigation before a full pass. | Symptom | Probable Defect | See Section | | Model "analyzes" its own reports/responses | Feedback loop | F | | Technical text, logs, code in "memories" | Input validation / scoring | A, D | | Same data repeats in context | Write idempotency | B | | Storage volume grew drastically after rebuild | Non-idempotent indexing | C | | Data exists in storage, retrieval is empty | Write/read key mismatch | 0, D | | "Works sometimes," "helps after restart" | Hidden stores, RAM state, cache | 0, C | | Database is "clean" but the problem persists | There is a store outside the map | 0 | | Old irrelevant displaces new | Static importance vs relevance | D | | Model suddenly "loses" all long-term memory | Non-deterministic persistence key | D | | Responses carry traces of unrelated topics/agents | Isolation boundary breach | G | | Short casual queries get empty context | Aggressive retrieval gate | D | | Response content is "cut off mid-word" | Blind truncation in assembly | E | | Discovered secrets surface in responses | Leak to memory before validation | A | | Hangs under load, "database is locked" | Access concurrency | H | | Agent "guesses" instead of "remembering" | Embedding model mismatch / retrieval path | 0, D | Audit Result Protocol Fill in after completing the pass: Verdict rules: - Failure of any CRIT in sections 0, A, B, E, F means: the architecture in its current form is unsafe for long-term memory accumulation — until remediation, data will be lost, poisoned, or reproduce garbage. - CRIT failures in C, D, G, H, I permit operation only with compensating controls monitoring, caller-side limits and a remediation plan. - The audit is valid until the next significant architectural change; after each change, affected sections are re-examined full pass — on architecture generation change . Note on the Standard's Origin This standard was derived by generalizing years of operational experience, incidents, and their post-mortems across real assistant systems with multi-layered memory: self-contamination loops, silent amnesia from non-deterministic keys, long-term storage poisoning by external content containing secrets, volume doubling from repeated indexing, embedding model language mismatches causing retrieval failures, degradation from simultaneous changes, personality trait loss on restart, and feedback loops from storing model outputs alongside user inputs. Each item is backed by a real incident of the corresponding class; items without an incident base are marked REC . Author: Aleksandr Kossarev, Jõgeva, Estonia Tags: ai architecture memory standard