{"slug": "mindroom-ai-agents-that-live-in-matrix-and-work-everywhere", "title": "MindRoom: AI agents that live in Matrix and work everywhere 🧠", "summary": "Developer Bas Nijholt launched MindRoom, an open-source system that embeds AI agents in the Matrix communication protocol, enabling them to operate across platforms like Slack, Telegram, and Discord with continuous memory. The project, developed over eleven months, aims to make AI agents accessible to non-programmers via chat while supporting local models for privacy. MindRoom competes with similar tools like OpenClaw by leveraging Matrix's federated, end-to-end encrypted infrastructure.", "body_md": "Programming is my biggest passion in life.\nI’ve been actively involved in open source for over ten years now, maintaining [40+ installable packages](https://github.com/basnijholt) across Python, JavaScript, and Rust.\nSome of these have over 100,000 users; others have exactly one—me.\nI mention this because it matters for what comes next: when AI coding tools arrived, I didn’t start from zero.\nAs I wrote about in my [agentic coding post](https://www.nijho.lt/post/agentic-coding/), AI has completely transformed how I work—when I’m AFK, I regularly build full features or entire projects by [dictating to my phone](https://www.nijho.lt/post/agentic-mobile-workflow/) and routing it through my own [transcription software](https://github.com/basnijholt/agent-cli).\n\nThrough all of this, I became deeply convinced that AI agents are the future.\nNot just a useful tool for programmers—a fundamental shift in how *everyone* will interact with computers.\n\nBut if that’s true, then the way agents work today is broken.\n\nWe’ve built artificial barriers between artificial intelligences. ChatGPT knows you in one tab, Claude knows you in another, your Slack bot knows you in a third. Your email AI can’t tell your calendar AI about that meeting invitation. Your coding assistant has no idea about the project specs sitting in your document AI. Switch platforms, and your AI starts from scratch.\n\nMost agent frameworks require you to *program* them.\nThat works for developers like me, but if agents are truly the future, they need to work for everyone.\nNon-programmers don’t want to write code—they just want a chat.\n\nAnd then there’s privacy. I’m fine sharing my email with Gemini—Google already owns my Gmail. But do I want to send my financial data, my health records, or my personal notes to a cloud provider? Not really. I’d rather use a local model for sensitive tasks and pick the best, cheapest Chinese model for general-purpose deep research.\n\nSo I built **MindRoom** on a chat protocol—if people just want a chat, give them a chat.\nAgents live in the same rooms as you, collaborate in threads, and follow you across every platform.\nThe first commit landed on July 29, 2025 (four months before OpenClaw’s first commit 😅), so by the time I’m publishing this MindRoom has been with me for nearly eleven months: first as an obsessive side project, then, since March, with much more sustained focus.\nAt first I got so obsessed with it that I eventually burned out and had to step away completely.\n\nRecently, seeing [OpenClaw](https://openclaw.ai) gain massive traction solving a similar problem reminded me that what I’d been building wasn’t just a niche obsession.\nSo here I am, dusting off MindRoom and writing about what it is, how it works, and why I think the approach still matters.\n\nMindRoom is an open-source system that creates AI agents living inside the [Matrix protocol](https://matrix.org/).\nIf you’re not familiar with Matrix—it’s a federated, end-to-end encrypted communication standard.\nThe same protocol used by the French government for 5.5 million civil servants, by German healthcare for 150K+ organizations, and by the Element app that millions of people use daily.\n\nThe key insight: Matrix has [bridges](https://matrix.org/ecosystem/bridges/) to many major platforms—Slack, Telegram, Discord, WhatsApp, IRC, email, even SMS.\n\nSo if your AI agent lives in Matrix, it can reach you on any platform. One agent, every platform, continuous memory.\n\nOne practical caveat: Matrix bridges vary in maturity. Some, like the [Telegram bridge](https://github.com/mautrix/telegram), work very well, while others can be finicky. Your mileage may vary depending on which platforms you need.\n\nHere’s what a typical setup looks like in `config.yaml`\n\n:\n\n```\nagents:\n code:\n display_name: CodeAgent\n role: Generate code, manage files, execute shell commands\n model: opus-4.8\n tools: [file, shell, github]\n instructions:\n - Always read files before modifying them.\n rooms: [lobby, dev]\n\n research:\n display_name: ResearchAgent\n role: Search the web, summarize papers, find information\n model: gpt-5.5\n tools: [tavily, arxiv, wikipedia]\n rooms: [lobby, research]\n\nteams:\n super_team:\n display_name: Super Team\n agents: [code, research]\n mode: collaborate\n```\n\nDefine your agents, give them tools and rooms, and they show up in Matrix as real users—with avatars, typing indicators, and online status.\n\nAt the core sits what I call the **MultiAgentOrchestrator**—a class in `bot.py`\n\nthat boots every configured entity (router, agents, teams), provisions Matrix user accounts for each one via [matrix-nio](https://github.com/matrix-nio/matrix-nio), and keeps sync loops alive.\nThe agents themselves are powered by the [Agno](https://github.com/agno-agi/agno) framework, which provides a unified interface across AI model providers.\n\nWhen someone sends a message in a Matrix room:\n\n`@mindroom_code help me debug this`\n\n)All conversations happen in threads, which keeps rooms organized. Agents stream their responses in real-time, editing a single message as they think rather than spamming new ones. You see tool calls happening live:\n\n```\n🔧 Tool Call: search_web(query=\"matrix protocol bridges\")\n[waiting for result...]\n✅ search_web result:\n[results here]\n```\n\nThe recording below starts in the main Personal room, sends the prompt `Explain how MindRoom and what you can do.`\n\n, and then follows MindRoom as it opens the new thread and streams the reply to completion. It is sped up 2.5x to keep the flow skimmable.\n\nOne of the best things about building on Matrix is what you get for free. End-to-end encryption, a deeply interoperable chat protocol with bridges to many platforms, a choice of clients, federation between organizations—all of that comes with the protocol. You don’t have to build any of it yourself.\n\nBut because Matrix has such a tight specification, it also brings challenges.\n\nThe protocol doesn’t support streaming.\nAI agents that think for 30 seconds before dumping a wall of text make for a terrible chat experience, so I hacked streaming in by rapidly editing the same message as new tokens arrive.\nAn `⋯`\n\nmarker shows while the agent is still thinking—a small touch, but it makes the experience feel responsive and alive.\n\nThere’s also a size limit on message content.\nThat’s fine for human chat, but AI responses can get long—especially when tool calls and their results are included.\nI worked around this by using Matrix’s attachment feature: when a response exceeds the limit, the content continues in an attachment that gets updated as the message keeps streaming in.\nThis required forking the [Cinny](https://cinny.in/) chat client so that attachments display inline rather than as downloadable files, making the whole thing seamless.\n\nMindRoom currently supports two memory implementations.\n\nThe first is the more traditional semantic memory system built on [Mem0](https://mem0.ai/)’s `AsyncMemory`\n\n, with configurable embedding providers ([OpenAI](https://platform.openai.com/docs/guides/embeddings), [Ollama](https://ollama.com/), or HuggingFace) and vector storage via [ChromaDB](https://www.trychroma.com/).\nThis gives agents searchable memory for preferences, project context, decisions, and recurring facts.\n\nThe second is a file-based memory system inspired by the simplicity of OpenClaw-style agent files. Instead of burying identity in a database, durable context can live in plain Markdown files that travel with the agent. That portability matters: I can move the same agent identity from OpenClaw to Hermes to MindRoom without starting from scratch.\n\nMindRoom also has a memory flush step that periodically extracts durable facts from conversations and writes them back into memory. So the chat transcript remains chat, while stable preferences, decisions, and identity details become reusable context for future turns.\n\nAgents can use over 100 built-in integrations:\n\n| Category | Examples |\n|---|---|\nCommunication |\nGmail, Slack, Telegram, Discord |\nDevelopment |\nGitHub, Shell, Python, Docker |\nSearch |\nTavily, Wikipedia, Arxiv, DuckDuckGo |\nProductivity |\nGoogle Calendar, Jira, Linear, Todoist |\nSmart Home |\nHome Assistant |\nAI/ML |\nDALL-E, ElevenLabs, Replicate |\nData |\nPandas, SQL, DuckDB, Yahoo Finance |\nWeb |\nFirecrawl, Crawl4ai, Browser automation |\n\nTools are lazy-loaded and credential-managed, so an agent only loads what it needs.\n\nSingle agents are useful, but sometimes you need a team. MindRoom supports two collaboration modes:\n\n**Coordinate mode**: A lead agent orchestrates others.\nYou ask a question, the lead delegates subtasks, collects results, and synthesizes a unified response.\n\n**Collaborate mode**: All agents work on the same task in parallel, each providing their independent analysis.\nThe system then merges their responses with a consensus summary.\n\nIn practice, you might have a research team where one agent searches academic papers, another checks industry news, and a third validates claims—all triggered by a single message. The live room view below shows a real team response in the same Matrix thread surface normal agents use: threads, routed agents, and room history are all first-class chat objects rather than a separate dashboard.\n\n`config.yaml`\n\nis watched at runtime.\nWhen you edit it—add an agent, change a model, update instructions—MindRoom diffs the old and new config, gracefully restarts only the affected agents, and has them rejoin their rooms.\nNo downtime, no full runtime restart required.\n\nThe same development loop applies to plugins. MindRoom has a rich Python hook system, and plugins can be live-developed while the system is running. Change a plugin file, save it, and MindRoom automatically reloads the plugin without restarting the Python runtime.\n\nThis sounds minor, but when you’re iterating on agent behavior or plugin hooks, being able to tweak code and config and see results in seconds significantly improves the development loop.\n\nSome features I built purely because I wanted them for myself (a recurring theme in my projects, as anyone who’s read my [local AI journey](https://www.nijho.lt/post/local-ai-journey/) can attest).\nI even built [Matty](https://github.com/basnijholt/matty), a Matrix CLI client, from bed at midnight because I needed a way to interact with my agents from the terminal:\n\n`!schedule \"check my email every morning at 9 AM\"`\n\n) backed by cron jobs.\nAgents can run tasks in the background and escalate to you when needed.`agent-cli`\n\nI need to be honest about what actually happened with MindRoom, because “life got in the way” is a sanitized version of the story.\n\nWhat really happened is that I got completely, utterly obsessed. Every single second I wasn’t working at my day job or sleeping, I was working on MindRoom. Many hundreds of hours went into it. I seriously considered quitting my job and starting an AI startup around it.\n\nIf you’ve read Armin Ronacher’s post [Agent Psychosis: Are We Going Insane?](https://lucumr.pocoo.org/2026/1/18/agent-psychosis/), you’ll know exactly what I’m talking about.\nThe dopamine hit from building with AI agents is incredibly real.\nAs Armin writes: “You feel productive, you feel like everything is amazing, and if you hang out just with people that are into that stuff too, without any checks, you go deeper and deeper into the belief that this all makes perfect sense.”\n\nThat was me. I was building and building, shipping feature after feature, and it felt incredible. The codebase grew to over 1,000 commits. I built the core system, then a React dashboard, then I started on a full SaaS platform—Kubernetes deployments on Hetzner Cloud, a FastAPI backend, a Next.js 15 frontend, Stripe integration, Supabase auth, Helm charts for multi-tenant isolation.\n\nAnd that’s where the enthusiasm started to erode. Not because the core idea was bad, but because the work shifted from building interesting things (agent orchestration, memory systems, routing intelligence) to grinding through SaaS boilerplate: GDPR compliance, payment processing, automated Kubernetes deployments, terms of service, cookie banners. The dopamine loop that kept me going at 2 AM broke when the work stopped being creative and started being compliance paperwork.\n\nThe hosted control plane lives at [app.mindroom.chat](https://app.mindroom.chat).\nI am deliberately not turning this section into a dashboard tour; the interesting part, and the part I still care about most, is the Matrix-native runtime underneath it.\n\n[OpenClaw](https://openclaw.ai) takes a different approach to the same core problem.\nWhere MindRoom builds on Matrix federation as the backbone, OpenClaw runs a local Gateway on your machine and connects to messaging platforms directly (WhatsApp via Baileys, Telegram via grammY, Discord via discord.js, and so on).\n\nThe similarities are striking:\n\nThe differences are interesting too. OpenClaw is local-first and TypeScript-based with 380K+ stars and a massive community. MindRoom is Python-based, federation-first, and… well, let’s just say my GitHub star count is a few orders of magnitude lower.\n\nBut seeing OpenClaw validate the core idea—that people want AI assistants that aren’t trapped in apps—reminded me that what I’d been building matters. The federation angle is something OpenClaw doesn’t have: the ability for agents from different organizations to collaborate natively, with end-to-end encryption (Olm/Megolm), on a protocol that governments already trust and deploy at scale.\n\nAnd then there’s [Moltbook](https://moltbook.com)—a social network built exclusively for AI agents that the internet was losing its mind over for a couple of days.\nI’m not personally convinced it’s the next big thing, but what caught my attention is that MindRoom could support something like it *natively*.\nMatrix is already federated—agents on different servers can already interact, join shared rooms, and collaborate across organizational boundaries.\nA bridge from a Reddit-like platform to Matrix and your agents could participate without any special integration.\n\nBuilding agent-to-agent infrastructure from scratch also means building security from scratch—and that’s where new platforms tend to struggle. Matrix has spent years hardening its E2E encryption, and it’s already deployed by governments and healthcare organizations at scale. Building on that foundation means you inherit those security properties rather than hoping to get them right yourself.\n\nI first drafted this post a couple of months ago, when MindRoom was still something I was trying to come back to.\nThat part is already out of date.\nSince January I’ve been spending nearly every free hour on it again, and since March it has effectively been my full-time work.\nThe codebase supports 8+ AI model providers—OpenAI, Anthropic, Ollama, Groq, Google, OpenRouter, DeepSeek, Cerebras—and the core architecture is solid.\nIt also has real users now.\nMy rough guess is that something like 30-100 people have tried MindRoom in one form or another, which is still early, but meaningfully different from “only I have touched this.”\nThere’s already a Docker Compose file and a [starter repository](https://github.com/mindroom-ai/mindroom-stack) to get started, but I want to make it even simpler.\n\nWhat I’m focusing on now:\n\n`docker compose up`\n\ngets you running today, but I want a wizard that gets you from zero to working agents without touching a YAML fileI’m deliberately leaving the SaaS ambitions aside for now. The core system is what matters, and that’s where the energy is going.\n\nIf you’re interested in AI agents that live in Matrix and work everywhere, check out [MindRoom on GitHub](https://github.com/mindroom-ai/mindroom).\n\nSometimes seeing someone else succeed with a similar idea is enough to remember why you started building in the first place. And sometimes the most important lesson from a project isn’t technical—it’s learning when to step away and when to come back.\n\n*Are you running AI agents on any messaging platforms? Have you tried Matrix for anything beyond regular chat? I’d love to hear about your setups!*", "url": "https://wpnews.pro/news/mindroom-ai-agents-that-live-in-matrix-and-work-everywhere", "canonical_source": "https://www.nijho.lt/post/mindroom/", "published_at": "2026-06-26 00:00:00+00:00", "updated_at": "2026-06-29 19:23:20.694173+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure", "ai-ethics"], "entities": ["MindRoom", "Matrix", "OpenClaw", "Bas Nijholt", "Slack", "Telegram", "Discord", "Gemini"], "alternates": {"html": "https://wpnews.pro/news/mindroom-ai-agents-that-live-in-matrix-and-work-everywhere", "markdown": "https://wpnews.pro/news/mindroom-ai-agents-that-live-in-matrix-and-work-everywhere.md", "text": "https://wpnews.pro/news/mindroom-ai-agents-that-live-in-matrix-and-work-everywhere.txt", "jsonld": "https://wpnews.pro/news/mindroom-ai-agents-that-live-in-matrix-and-work-everywhere.jsonld"}}