{"slug": "seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory", "title": "Seeking AI/ML Collaborators: Building Marven — A Local AI with Persistent Memory, Continual Learning, and Human-State Awareness", "summary": "Heshware LLC founder is developing Marven, a local-first AI assistant designed to maintain persistent memory, continual learning, and human-state awareness while keeping user data under their control. The project explores layered memory architectures and a MetaMirror system to distinguish important memories from trivial ones, aiming for a Personal Sovereign AI that operates on user-owned infrastructure.", "body_md": "Hi everyone,\n\nI’m the founder of [Heshware LLC](https://heshware.it.com), and I’m currently developing an AI project called **Marven**.\n\nMarven started as an experiment in building a more personal AI assistant, but over time the project has grown into a larger research and engineering question:\n\n**What would it take to build a local-first AI system that doesn’t simply respond to prompts, but develops meaningful continuity with a user over long periods of time?**\n\nThe goal is not to claim that Marven is conscious or to build another wrapper around an LLM.\n\nI’m interested in developing an architecture where an AI can maintain persistent memory, understand changing human context, learn from long-term interactions, preserve continuity across sessions, and operate primarily on infrastructure controlled by the user.\n\n**What I’m Trying to Build**\n\nThe long-term vision for Marven is a **Personal Sovereign AI** — an AI system whose memory, models, identity data, and personal context can remain under the user’s control rather than existing entirely inside a centralized cloud platform.\n\nSome of the capabilities I’m exploring include:\n\nOne of the core questions behind Marven is:\n\n**Can an AI maintain continuity with someone for years while still giving that person ownership and control over what the AI remembers?**\n\nMarven is still experimental and the architecture continues to evolve.\n\nThe current/local stack has included:\n\n**LLM inference**\n\n**Memory**\n\nI’ve also been experimenting with what I call the **MetaMirror** system.\n\nMetaMirror is intended to represent more than conversation history. The idea is to maintain structured observations about interaction context and how that context changes over time.\n\nFor example:\n\n```\nInteraction\n   ↓\nConversation / Voice Input\n   ↓\nContext Analysis\n   ↓\nWorking Memory\n   ↓\nMemory Importance Evaluation\n   ↓\nEpisodic / Semantic Memory\n   ↓\nLong-Term Memory Archive\n   ↓\nRetrieval + Context Reconstruction\n   ↓\nFuture Marven Interaction\n```\n\nThe architecture is moving toward separating memory into different layers instead of continuously dumping conversations into a vector database.\n\nThat distinction is becoming increasingly important to me.\n\nRemembering everything is not necessarily intelligence.\n\nKnowing **what should be remembered, what should be forgotten, what should be summarized, and when something should be retrieved** is a much harder problem.\n\nAnother part of the project is what I’m currently thinking of as a **Human-State Engine**.\n\nThe idea is to give Marven additional context regarding *how* someone is communicating rather than only processing the literal words they say.\n\nPotential signals include:\n\nFuture experiments could potentially include visual signals, but I want this architecture to be privacy-conscious and based around explicit user consent.\n\nFor example, instead of simply receiving:\n\n```\n\"I'm fine.\"\n```\n\nthe system might internally receive contextual information closer to:\n\n```\nText: \"I'm fine.\"\n\nVoice indicators:\n- lower energy than personal baseline\n- longer pause before response\n- reduced speaking speed\n\nConversation context:\n- previously discussing stressful event\n\nConfidence:\n- uncertain emotional state\n```\n\nThe LLM still determines how to respond, but it receives richer contextual information.\n\nImportantly, I do **not** want Marven confidently diagnosing someone’s emotions.\n\nHuman-state detection should be probabilistic and uncertain by design.\n\nI’ve also been working on a persistent visual representation of Marven.\n\nThe current direction uses a 3D digital-human interface with states such as:\n\nThe goal isn’t simply to create an animated chatbot.\n\nI’m interested in whether maintaining a consistent visual and behavioral presence contributes to the feeling of continuity in long-term human-AI interaction.\n\nThe voice pipeline has experimented with technologies including:\n\nEventually I want voice interaction to operate with sufficiently low latency that Marven feels less like:\n\n**Speak → wait → AI responds**\n\nand more like a natural conversational system.\n\nThat requires work across:\n\n```\nWake word\n   ↓\nVoice activity detection\n   ↓\nStreaming STT\n   ↓\nContext + memory retrieval\n   ↓\nModel inference\n   ↓\nStreaming TTS\n   ↓\nDigital-human animation\n```\n\nLatency across that entire pipeline is one of the engineering challenges I’m currently investigating.\n\nThe part of Marven I’m currently most interested in improving is the memory architecture.\n\nMost current AI assistants effectively have some combination of:\n\n```\nConversation History\n+\nVector Database\n+\nRetrieval\n```\n\nThat works, but I’m interested in something more structured.\n\nI’m experimenting conceptually with a system closer to:\n\n```\nSensory / Interaction Memory\n          ↓\n     Working Memory\n          ↓\n   Importance Scoring\n          ↓\n ┌────────┴─────────┐\n ↓                  ↓\nEpisodic          Semantic\nMemory             Memory\n ↓                  ↓\n └────────┬─────────┘\n          ↓\nMemory Consolidation\n          ↓\nLong-Term Archive\n          ↓\nContextual Retrieval\n```\n\nSome questions I’m investigating:\n\n**How should memory importance be calculated?**\n\nSomething mentioned once may be extremely important, while something repeated every day may not be.\n\n**How should memory decay work?**\n\nHuman memory isn’t an infinitely growing database.\n\n**How should contradictions be handled?**\n\nIf someone’s preferences or beliefs change, the AI shouldn’t continue retrieving outdated information as if it were current.\n\n**How do you prevent memory poisoning?**\n\nIf an attacker can manipulate persistent memory, prompt injection becomes much more dangerous.\n\n**How should memories be consolidated?**\n\nHundreds of interactions about the same subject should potentially become a higher-level semantic representation rather than hundreds of individual memories.\n\n**How can the model know why a memory matters?**\n\nSimilarity alone may not be enough for good retrieval.\n\nAnother challenge is determining how much of Marven’s learning should happen through:\n\nI don’t think continuously fine-tuning the base model on every user interaction is necessarily the correct solution.\n\nMy current thinking is that most personalization should initially happen through memory and contextual systems, while actual model-weight updates happen much more deliberately.\n\nI’d be especially interested in hearing from anyone researching continual learning or catastrophic forgetting.\n\nPersistent memory also creates an entirely new security problem.\n\nI’m actively thinking about threats including:\n\nA personal AI could eventually know more sensitive contextual information about someone than almost any other piece of software they use.\n\nThat means security cannot be something added after the architecture is finished.\n\nI’m particularly interested in architectures where memories have permissions, provenance, confidence scores, timestamps, and potentially cryptographic integrity verification.\n\nOne of Marven’s fundamental design principles is that users should eventually have the option to run significant portions of the system locally.\n\nThat could mean:\n\n```\nPersonal Device\n       ↓\nLocal Marven Node\n       ↓\nLocal Memory Database\n       ↓\nLocal / Home LLM Server\n       ↓\nOptional Cloud Models\n```\n\nThe cloud should ideally become an optional compute resource rather than the permanent owner of someone’s AI memory.\n\nI’m also exploring home-server/NAS-style architectures where an individual’s long-term AI memory could exist on storage they physically control.\n\nMarven isn’t only a concept.\n\nI’ve already experimented with or built portions of:\n\nA lot of these pieces are currently separate experiments.\n\nOne of my current priorities is turning them into a cleaner unified architecture.\n\nI’m posting here because I would really like feedback from people who work in areas such as:\n\nI’m especially interested in contributors who enjoy experimenting with research ideas and turning them into working prototypes.\n\nYou don’t need to be an expert in every area.\n\nEven feedback like:\n\n“Your memory architecture will eventually break because of X.”\n\nor\n\n“There’s a paper/project you should study that already tackles part of this.”\n\nwould be extremely useful.\n\nI’d particularly like opinions on a few questions:\n\n**1. If you were designing long-term memory for an AI intended to interact with someone for 5–10+ years, what architecture would you use?**\n\n**2. Would you combine vector retrieval with a knowledge graph, or use another architecture entirely?**\n\n**3. What is the best way to separate episodic memory from semantic/user-profile memory?**\n\n**4. How would you design memory consolidation and forgetting?**\n\n**5. How would you defend persistent AI memory against poisoning and prompt injection?**\n\n**6. What parts of personalization actually deserve model fine-tuning versus external memory?**\n\n**7. Are there open-source projects or papers exploring similar long-duration human-AI relationships that you think I should study?**\n\nI’m also open to collaborating with developers or researchers who find these problems interesting.\n\nRight now I’m particularly interested in people who could help experiment with:\n\n**Memory / RAG**\n\n**Local AI**\n\n**Continual Learning**\n\n**Voice / Multimodal**\n\n**AI Security**\n\nThis is still an early-stage research and development project, so I’m interested in people who enjoy exploring difficult questions rather than only implementing a predetermined product specification.\n\nMarven is being developed through **Heshware LLC**, but I’m also interested in keeping portions of the research and experimentation open enough that other developers can learn from it and contribute ideas.\n\nUltimately, the question I’m trying to explore with Marven is pretty simple:\n\n**What happens when an AI stops being something you repeatedly start a new conversation with and instead becomes a system capable of maintaining years of meaningful, user-controlled continuity?**\n\nThere are a lot of unanswered engineering, security, privacy, and research questions between here and there.\n\nThat’s exactly why I’m interested in building it.\n\nI’d love to hear what the Hugging Face community thinks.", "url": "https://wpnews.pro/news/seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory", "canonical_source": "https://discuss.huggingface.co/t/seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory-continual-learning-and-human-state-awareness/179422#post_1", "published_at": "2026-08-28 21:24:36+00:00", "updated_at": "2026-08-28 21:48:10.039783+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-products", "ai-infrastructure"], "entities": ["Heshware LLC", "Marven", "MetaMirror"], "alternates": {"html": "https://wpnews.pro/news/seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory", "markdown": "https://wpnews.pro/news/seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory.md", "text": "https://wpnews.pro/news/seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory.txt", "jsonld": "https://wpnews.pro/news/seeking-ai-ml-collaborators-building-marven-a-local-ai-with-persistent-memory.jsonld"}}