{"slug": "i-built-an-ai-security-assistant-that-remembers-previous-investigations", "title": "I Built an AI Security Assistant That Remembers Previous Investigations", "summary": "A developer built ThreatMemory, an AI security alert triage assistant that adds a persistent memory layer to LLM-based analysis so past investigations inform new alerts. The system uses Hindsight to retrieve relevant historical cases and analyst decisions, feeding them to the LLM as context while explicitly instructing it not to blindly copy prior verdicts, with the analyst retaining final decision authority.", "body_md": "Security alerts are rarely completely new.\n\nA security analyst might see dozens of failed-login alerts, suspicious IP addresses, unusual data transfers, or large file movements. Many of these incidents resemble cases the team has already investigated.\n\nThe problem is that a typical LLM starts each analysis from scratch.\n\nIt can understand the alert in front of it, but it doesn't automatically know how the security team handled a similar incident last week.\n\nI built ThreatMemory to explore a different approach: an AI security alert triage assistant with persistent memory.\n\nThe key idea is simple:\n\nInstead of only asking an AI what it thinks about an alert, let it remember what the security team learned from previous alerts.\n\nThe problem with stateless alert analysis\n\nConsider a security alert like this:\n\n36 failed authentication attempts against [payroll@company.com](mailto:payroll@company.com) from IP 185.20.4.21 between 01:50 AM and 02:05 AM.\n\nA normal AI assistant can analyze the information contained in that alert.\n\nIt might identify several possibilities:\n\nA legitimate employee repeatedly entering the wrong password\n\nA VPN-related authentication problem\n\nA brute-force attempt\n\nCredential stuffing\n\nA compromised device\n\nWithout additional context, the AI has no way to know how this organization's security team handled similar events previously.\n\nThat means every alert becomes a new investigation.\n\nThreatMemory adds a memory layer to this process.\n\nThe architecture\n\nThe application has three main components:\n\nSecurity analyst → ThreatMemory → Hindsight + LLM\n\nThe workflow is:\n\nThe analyst provides a security alert.\n\nThreatMemory sends the alert to Hindsight for relevant historical cases.\n\nHindsight recalls previous investigations and analyst decisions.\n\nThe retrieved cases are provided to the LLM as context.\n\nThe LLM produces a recommendation and investigation steps.\n\nThe analyst makes the final decision.\n\nThe analyst's decision and reasoning are retained in Hindsight.\n\nFuture alerts can retrieve that experience.\n\nThis creates a continuous loop:\n\nAlert → Recall → Analyze → Analyst Decision → Retain → Future Recall\n\nHindsight is therefore not just another component in the application. It is the part that allows previous investigations to become usable context for future ones.\n\nThe before-and-after difference\n\nThe most important part of ThreatMemory is the difference between analyzing an alert with memory and without memory.\n\nMemory OFF\n\nWhen Hindsight memory is disabled, ThreatMemory explicitly tells the LLM:\n\nMemory is OFF.\n\nDo not use any historical cases.\n\nAnalyze this alert only from the information contained in the alert itself.\n\nFor the example alert, the AI identified the unusually high number of failed attempts and recommended further investigation.\n\nThat is reasonable.\n\nBut it is working with only the current alert.\n\nMemory ON\n\nNow the same alert is analyzed with Hindsight enabled.\n\nThreatMemory retrieves relevant historical investigations.\n\nFor example, previous cases may show that similar failed-login alerts originated from the organization's corporate VPN infrastructure and were ultimately classified as false alarms.\n\nThe LLM can now consider that history alongside the current alert.\n\nInstead of starting from zero, it has organizational context.\n\nThe important distinction is that ThreatMemory does not blindly copy an old decision.\n\nThe prompt explicitly tells the model:\n\nWhen historical cases are provided, use them as context.\n\nMemory should influence the recommendation naturally.\n\nDo not blindly copy an old decision.\n\nThe analyst always makes the final decision.\n\nThis matters because a similar-looking alert can still represent a completely different incident.\n\nHow Hindsight is used\n\nThe core memory operation is retrieval.\n\nConceptually, ThreatMemory takes the current alert and asks Hindsight:\n\n“What previous investigations are relevant to this alert?”\n\nThe application then passes the retrieved cases to the LLM as historical context.\n\nA simplified part of the implementation looks like this:\n\nif memory_enabled and memories:\n\n    memory_text = \"\\n\\n\".join(\n\n        f\"PAST CASE {i + 1}:\\n{m['text']}\"\n\n        for i, m in enumerate(memories)\n\n    )\n\n```\ncontext = f\"\"\"\n```\n\nHistorical security cases retrieved from Hindsight:\n\n{memory_text}\n\n\"\"\"\n\nThe important part is that the model isn't receiving an arbitrary collection of documents.\n\nThe historical cases are retrieved specifically in response to the current alert.\n\nThat allows semantically similar incidents to become relevant even when the wording or exact details differ.\n\nMemory doesn't replace the analyst\n\nOne design decision was important from the beginning:\n\nThreatMemory is decision support, not an autonomous security system.\n\nThe AI provides:\n\nA recommendation\n\nConfidence\n\nReasoning\n\nRecommended investigation steps\n\nRelevant historical cases\n\nBut the analyst makes the final decision.\n\nThe interface contains a dedicated Analyst Decision section with three options:\n\nFalse Alarm\n\nReal Threat\n\nNeeds Investigation\n\nThe analyst can also provide a reason for the decision.\n\nThat decision is then stored back into Hindsight.\n\nFor example:\n\nThe source IP was confirmed as corporate VPN infrastructure and the employee verified the login attempts as legitimate.\n\nThis transforms an analyst's investigation from a one-time action into future context.\n\nThe learning loop\n\nThis is where the system becomes more interesting than a simple RAG application.\n\nSuppose an analyst investigates an alert and discovers that it was a legitimate VPN event.\n\nThreatMemory stores that outcome.\n\nLater, another alert appears with similar characteristics.\n\nHindsight can retrieve the previous investigation, allowing the AI to consider the team's previous experience.\n\nThe system therefore has two directions of memory:\n\nRecall:\n\n“What have we learned about cases like this?”\n\nRetain:\n\n“What did the analyst learn from this case?”\n\nOver time, the memory store can contain different types of experience:\n\nRepeated false alarms\n\nConfirmed attacks\n\nLegitimate backup activity\n\nBusiness-travel login events\n\nAnalyst overrides\n\nPreviously unknown patterns that were later resolved\n\nThis makes the memory useful beyond simply remembering conversations.\n\nWhat I learned building it\n\nThe biggest lesson was that adding memory isn't automatically useful.\n\nThe memory has to affect the actual workflow.\n\nA system that retrieves five old records but doesn't change how the current task is handled isn't meaningfully using agent memory.\n\nFor ThreatMemory, the memory layer is directly connected to the decision process:\n\nCurrent alert → Relevant experience → AI reasoning → Analyst decision → New experience\n\nAnother important lesson was to keep the scope narrow.\n\nInstead of trying to build a complete security operations platform, ThreatMemory focuses on one workflow: security alert triage.\n\nThat makes the role of memory easy to understand and easy to demonstrate.\n\nA limitation\n\nThreatMemory is currently a prototype using realistic synthetic security cases.\n\nThat means its historical memory is only as useful as the information stored in it.\n\nA production system would need much stronger safeguards around data quality, access control, privacy, retention policies, auditability, and the accuracy of analyst decisions.\n\nIt would also need integration with real security systems such as authentication logs, SIEM platforms, endpoint telemetry, and incident-management systems.\n\nThe current application deliberately stops before autonomous response.\n\nIt recommends.\n\nThe analyst decides.\n\n🔗 **Live Demo:** [https://threatmemory-f2kdfnwvpefa9isiquicgxv.streamlit.app](https://threatmemory-f2kdfnwvpefa9isiquicgxv.streamlit.app)\n\n💻 **GitHub Repository:** [https://github.com/adithya9666/threatmemory](https://github.com/adithya9666/threatmemory)\n\nWhat's next\n\nThe next step would be to connect ThreatMemory to real organizational security data and allow the memory to grow naturally from real investigations.\n\nThe broader idea goes beyond cybersecurity.\n\nMany professional workflows have the same problem: people repeatedly make decisions using information that their organization has already learned, but that knowledge is scattered across previous cases.\n\nPersistent agent memory creates a way for AI systems to carry that experience forward.\n\nFor ThreatMemory, the goal is straightforward:\n\nDon't make the analyst investigate every alert as if it has never happened before.\n\nGive the AI access to what the team has already learned — while keeping the human analyst in control.", "url": "https://wpnews.pro/news/i-built-an-ai-security-assistant-that-remembers-previous-investigations", "canonical_source": "https://dev.to/adithya9666/i-built-an-ai-security-assistant-that-remembers-previous-investigations-hg4", "published_at": "2026-09-28 19:37:36+00:00", "updated_at": "2026-09-28 19:50:11.185191+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools"], "entities": ["ThreatMemory", "Hindsight"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-built-an-ai-security-assistant-that-remembers-previous-investigations", "markdown": "https://wpnews.pro/news/i-built-an-ai-security-assistant-that-remembers-previous-investigations.md", "text": "https://wpnews.pro/news/i-built-an-ai-security-assistant-that-remembers-previous-investigations.txt", "jsonld": "https://wpnews.pro/news/i-built-an-ai-security-assistant-that-remembers-previous-investigations.jsonld"}}