Build a Privacy Filter Before Your AI Agent Remembers User Actions

A developer outlines a privacy filter for AI agents that record user actions, emphasizing the need to capture minimal event streams and apply strict redaction and retention policies. The guide proposes a five-job filter covering event allowlisting, sensitive data detection, purpose binding, retention, and retrieval control, with a TypeScript example for implementation.

AI agents are starting to remember more than chats. They can watch clicks, typed text, app switches, browser context, files, tool calls, and workflow history. That memory can make an agent feel useful fast, but it can also turn a helpful feature into a quiet privacy incident. If you are building an AI product, do not start with “how much can we capture?” Start with “what is the smallest event stream that still helps the user?” This guide shows a practical privacy filter you can place between raw user activity and agent memory. Recent AI tooling trends point in the same direction: agents are moving from chat boxes into operating systems, browsers, IDEs, customer support tools, analytics dashboards, and workflow automation platforms. The more useful the agent becomes, the more context it wants. That creates a new engineering problem. Traditional app logs record requests and errors. Agent memory records intent, context, and behavior. A raw event can include: This is not just observability. It is a privacy boundary. The practical trigger is simple: computer-use agents and workflow agents now need history to resume work, personalize answers, and automate multi-step tasks. But developers, security reviewers, and buyers are asking harder questions about PII, retention, auditability, user consent, and whether agent traces can leak private business data. Most teams already have logs, traces, analytics events, and support transcripts. So when they add agent memory, they often reuse the same pattern: That is easy to ship. It is also too broad. Agent memory needs a stricter path because it may be used to generate future answers or actions. A normal log line might be seen by engineers. A memory item might be read by a model, combined with other data, and used to make a decision. The safer pattern is: Raw event → privacy filter → purpose check → redacted memory → retention policy → retrieval policy The privacy filter is not a prompt. It is application code that decides what the agent is allowed to remember. A good privacy filter has five jobs. | Job | Question it answers | Example | |---|---|---| | Event allowlist | Should this event be captured at all? | Save “opened invoice page,” not every mouse coordinate. | | Sensitive data detection | Does the payload contain PII, secrets, or regulated data? | Detect emails, API keys, card-like numbers, tokens. | | Purpose binding | Why is this memory needed? | Resume task, improve support, audit approval. | | Retention control | How long can this memory live? | 48 hours for raw traces, 30 days for redacted task summaries. | | Retrieval control | Who or what can read it later? | Only the same user, tenant, role, and task type. | The filter should run before indexing, summarization, embedding, analytics export, or model calls. Do not start with redaction. Start with event classes. Redaction helps when you must keep data. Classification helps you avoid collecting data in the first place. A simple event taxonomy might look like this: | Event class | Risk | Store by default? | Notes | |---|---|---|---| | Navigation event | Low | Yes, redacted | Page type, not full URL if it contains IDs. | | Tool call metadata | Medium | Yes | Store tool name, status, cost, policy result. | | User typed text | High | No | Store only if explicitly needed and redacted. | | Screen content | High | No | Prefer structured app state over screenshots. | | File content | High | No | Store references and hashes, not full content. | | Approval decision | Medium | Yes | Keep reviewer, action, timestamp, and reason. | | Secret or credential | Critical | Never | Block and alert if detected. | Here is a small TypeScript example: type EventClass = | "navigation" | "tool call" | "typed text" | "screen content" | "file content" | "approval" | "secret"; type CaptureDecision = "store" | "redact then store" | "summarize only" | "drop"; const capturePolicy: Record<EventClass, CaptureDecision = { navigation: "redact then store", tool call: "store", typed text: "summarize only", screen content: "drop", file content: "summarize only", approval: "store", secret: "drop", }; This looks boring. That is the point. Privacy should not depend on a clever prompt at runtime. A raw event often contains too much context. Reduce it into a smaller shape before running PII detection. Bad memory candidate: { "type": "typed text", "value": "My card is 4242 4242 4242 4242 and my email is alex@example.com", "url": "https://app.example.com/customers/cus 782/orders/ord 991", "dom": "...full page text...", "timestamp": "2026-08-15T03:30:00Z" } Better memory candidate: { "type": "task signal", "summary": "User entered payment-related information during checkout setup.", "page type": "checkout settings", "tenant id": "tenant 123", "user id": "user 456", "timestamp": "2026-08-15T03:30:00Z" } Notice what changed: Reduction is the cheapest privacy win you can ship. Use multiple detectors. Regex is not enough, but regex is still useful. You want to detect: Example filter: type Redaction = { redacted: string; findings: Array<{ type: string; count: number } ; }; function redactSensitiveText input: string : Redaction { const findings: Redaction "findings" = ; let text = input; const patterns = { type: "email", regex: / A-Z0-9. %+- +@ A-Z0-9.- +\. A-Z {2,}/gi }, { type: "credit card like", regex: /\b ?:\d - ? {13,19}\b/g }, { type: "api key like", regex: /\b ?:sk|pk|ghp|xoxb|AKIA A-Za-z0-9 \- {16,}\b/g }, { type: "private key", regex: /-----BEGIN A-Z PRIVATE KEY----- \s\S ?-----END A-Z PRIVATE KEY-----/g }, ; for const pattern of patterns { const matches = text.match pattern.regex ; if matches?.length { findings.push { type: pattern.type, count: matches.length } ; text = text.replace pattern.regex, REDACTED ${pattern.type.toUpperCase } ; } } return { redacted: text, findings }; } For production, combine this with a structured PII service, domain-specific dictionaries, and field-level policies. The key is to store the findings separately from the raw value. A memory without a purpose becomes a future liability. Add a purpose field when the memory is created. Common purposes: resume task support debugging security audit billing dispute quality evaluation personalization Each purpose should control retention and retrieval. type MemoryPurpose = | "resume task" | "support debugging" | "security audit" | "billing dispute" | "quality evaluation" | "personalization"; const retentionDays: Record<MemoryPurpose, number = { resume task: 7, support debugging: 30, security audit: 180, billing dispute: 365, quality evaluation: 14, personalization: 90, }; This gives your product, legal, and engineering teams one shared control surface. It also prevents a common failure mode: using data collected for debugging as long-term personalization memory. Raw traces and agent memory should not live in the same bucket. A useful split: Raw event buffer Short-lived, encrypted, tightly restricted, used for immediate debugging or user-visible replay. Redacted memory store Longer-lived, purpose-bound, searchable by the agent only through policy checks. Audit ledger Append-only records of decisions: what was stored, why, which policy allowed it, and when it expires. The agent should usually retrieve from the redacted memory store, not the raw buffer. User activity | v Raw event buffer: short TTL | v Privacy filter | +-- Drop / block / alert | v Redacted memory store | v Policy-checked retrieval | v Agent response or action This structure makes deletion easier and audits less painful. Consent is not a checkbox on the settings page. It is runtime state. Check consent when: Example: type ConsentState = { userId: string; tenantId: string; allowAgentMemory: boolean; allowPersonalization: boolean; allowSupportReview: boolean; revokedAt?: string; }; function canStoreMemory consent: ConsentState, purpose: MemoryPurpose : boolean { if consent.allowAgentMemory || consent.revokedAt return false; if purpose === "personalization" return consent.allowPersonalization; if purpose === "support debugging" return consent.allowSupportReview; return true; } If consent is revoked, new memory should stop immediately. Existing memory should either expire, be deleted, or become inaccessible depending on your product policy and legal requirements. A memory can be safe to store but unsafe to retrieve in a different context. Before retrieving memory for an agent, check: This prevents awkward bugs like a support agent retrieving billing context during a product tutorial, or a workspace agent pulling private notes into a shared channel. A retrieval policy can be simple: function canRetrieveMemory args: { requesterTenantId: string; requesterUserId: string; memoryTenantId: string; memoryUserId: string; purpose: MemoryPurpose; requestedPurpose: MemoryPurpose; expiresAt: Date; } { if args.requesterTenantId == args.memoryTenantId return false; if args.requesterUserId == args.memoryUserId return false; if args.purpose == args.requestedPurpose return false; if args.expiresAt.getTime < Date.now return false; return true; } In team products, replace the user equality check with a role and resource policy. Keep the default narrow. You still need audit logs. Just do not put raw secrets in them. A good audit record includes: { "memory id": "mem 123", "tenant id": "tenant 123", "user id": "user 456", "event class": "typed text", "decision": "summarize only", "purpose": "resume task", "pii findings": { "type": "email", "count": 1 } , "policy version": "privacy-filter-v4", "created at": "2026-08-15T03:30:00Z", "expires at": "2026-08-22T03:30:00Z" } This record helps you answer: That is much better than saving the whole raw payload and hoping nobody looks too closely. Use this as a build checklist for your first privacy filter. drop or summarize only .Most privacy guides cover PII redaction, audit logs, or broad data governance. Agent memory needs one more layer: event design. Decide what to do with clicks, typed text, page context, tool calls, desktop actions, and model-readable summaries before they enter storage. Ask this before saving anything: Would this memory still feel reasonable if the user inspected it, exported it, or saw it during an incident review? If the answer feels uncomfortable, reduce it. For a small team, do not overbuild. Start with this: agent events table with short retention. agent memories table with redacted summaries only. agent memory audit table for policy decisions.Schema sketch: CREATE TABLE agent memories id TEXT PRIMARY KEY, tenant id TEXT NOT NULL, user id TEXT NOT NULL, purpose TEXT NOT NULL, sensitivity TEXT NOT NULL, summary TEXT NOT NULL, metadata JSONB NOT NULL DEFAULT '{}', policy version TEXT NOT NULL, created at TIMESTAMPTZ NOT NULL DEFAULT now , expires at TIMESTAMPTZ NOT NULL ; CREATE INDEX agent memories lookup ON agent memories tenant id, user id, purpose, expires at ; Then test it like a security feature, not like a logging feature. Add these to CI: These tests will catch more real issues than another paragraph in your privacy policy. Agent memory is powerful because it compresses user context into future usefulness. That same compression can hide privacy mistakes if you capture too much, store it too long, or retrieve it in the wrong place. The safest path is not “never remember anything.” That makes agents weak. The safest path is to remember less, explain why, expire it on purpose, and retrieve it only when the current task deserves it. Build the privacy filter before the memory feature becomes popular. It is much easier to start narrow than to clean up a giant pile of raw history later. An AI agent privacy filter is application logic that decides which user activity events can become agent memory. It classifies events, redacts sensitive data, checks consent, assigns a purpose, applies retention, and controls retrieval. Usually no. Raw actions such as typed text, full page content, screenshots, and file contents are high risk. Store reduced summaries, task state, tool metadata, or redacted memory instead. Logs are mainly used for debugging and operations. Agent memory may be retrieved by a model and used to generate future responses or actions. That makes purpose, consent, retention, and retrieval policy more important. Do not store raw secrets, API keys, session cookies, private keys, payment details, or regulated personal data unless you have a very specific, compliant reason. In most products, these should be blocked or heavily redacted. Retention depends on purpose. Task-resume memory may only need days. Support debugging may need weeks. Security audit records may need longer. Avoid one global retention window for every memory type. No. Prompts can remind a model not to reveal sensitive data, but privacy enforcement should happen in code before storage, indexing, embedding, and retrieval. Give users visibility and control. Provide settings to disable memory, inspect saved memory, delete memory, and understand what the agent remembers and why.