I Built an Open-Source Studio for Building, Testing, and Deploying AI Agents

A developer has released Chatbot Studio, an MIT-licensed, self-hostable open-source platform for building, testing, evaluating, and deploying AI agents as website chatbots or channel integrations such as WhatsApp. The project separates the agent (model, knowledge, tools, memory, guardrails) from its presentation layer, so a single agent remains the source of truth across multiple published chatbots, and it supports Model Context Protocol (MCP) servers, reusable knowledge bases, and multiple model providers.

Building an AI chatbot is easy. Shipping one that has tools, knowledge, memory, observability, evaluations, human handoff, multiple model providers, workflows, and an actual interface your users can interact with is a different problem. That gap is what led me to build Chatbot Studio . It's an open-source platform for building AI agents and then publishing those agents as website chatbots or connecting them to channels such as WhatsApp. This isn't a SaaS announcement or a sales pitch. The project is MIT licensed, self-hostable, and available on GitHub: GitHub: https://github.com/judejulius/ChatbotStudio https://github.com/judejulius/ChatbotStudio A lot of AI projects begin with something like this: response = client.chat.completions.create model="...", messages= ... Then reality arrives. You need tools. Then retrieval. Then credentials. Then streaming. Then conversation state. Then rate limits. Then evaluations. Then traces because something went wrong in production. Then a UI. Then somebody asks: "Can we put this on the website?" And someone else asks: "Can customers talk to a human if the AI gets stuck?" At that point you're no longer building a prompt around an LLM. You're building an agent platform . That is the problem Chatbot Studio is trying to explore. One architectural decision became especially important while building this. An agent should not be the same object as its presentation layer. The agent owns things such as: A published chatbot owns the things that belong to the channel: In other words: Model + Knowledge + Tools | v Agent | test / evaluate | v Published Chatbot | Website / Channel The chatbot isn't a duplicated agent configuration. It's a channel sitting on top of the agent. That means I can improve an agent's instructions, knowledge, model, or tools without recreating every chatbot using it. The agent remains the source of truth. The main Agent Studio lets you configure and test an agent before publishing it. Currently the platform supports provider families including: An agent can then be connected to knowledge bases, tools, MCP servers, memory, skills, guardrails, and other runtime controls. The test chat runs against the saved agent configuration. That part matters. I didn't want a playground where the test environment was secretly different from the thing that eventually gets deployed. The goal is: php configure - test - evaluate - publish rather than: php prototype - rewrite everything - deploy something different One part I've been particularly interested in is Model Context Protocol support. Chatbot Studio can connect agents to MCP servers using: This makes external capabilities much easier to attach to an agent without baking every integration directly into the application. The broader architecture becomes something like: +----------------+ | Knowledge Base | +-------+--------+ | +----------+ +------v------+ | MCP Tool +----- + | +----------+ | Agent | | | +----------+ +------+------+ | API Tool +------------+ +----------+ | v Conversation For me, MCP is interesting because it moves agent tooling toward a more interoperable ecosystem instead of every project inventing its own tool interface. Agents can also be connected to reusable knowledge bases. The backend currently handles sources including: Documents are processed into chunks and embeddings that can be retrieved during conversations. The important idea here was making knowledge a reusable platform resource rather than stuffing documents directly into one chatbot implementation. One knowledge base can therefore become part of a broader agent configuration. For deployment on websites, I wanted the exported chatbot to be as framework-independent as possible. So the primary integration is a browser-native custom element: <script type="module" src="https://your-domain.com/widget.esm.js?widget=wgt ..." </script <chatbot-widget </chatbot-widget The component uses a Shadow DOM so the host website's CSS doesn't unexpectedly destroy the chatbot UI—and the chatbot doesn't leak its styles back into the host application. The element also exposes a small JavaScript API: js const chatbot = document.querySelector "chatbot-widget" ; chatbot.open ; chatbot.send "I need help with an order" ; chatbot.close ; chatbot.reset ; And it emits events such as: chatbot-ready chatbot-error chatbot-handoff Because the core is a Web Component, framework integrations can stay fairly thin. Chatbot Studio can generate integrations for: Instead of maintaining five completely different chatbot implementations, they all revolve around the same browser-native element. I also wanted customization to use the real renderer . The editor therefore mounts the same widget implementation that gets exported. You can customize things such as: This avoids a problem I've seen in visual builders where the editor preview looks one way and the actual embedded component behaves differently. The preview and the shipped widget share the same rendering path. AI shouldn't have to pretend it can solve every problem. So Chatbot Studio also has human handoff. Instead of handing a conversation off to an undefined "human", conversations can be routed into queues such as: General Support Technical Support Sales Billing Administrators control which users belong to which queues. The AI can hand the conversation to an appropriate queue, after which a person can take ownership of it. Once handed over, the assistant stops trying to answer the conversation as though nothing happened. This required treating handoff as application state rather than just another tool response. Not every task belongs inside one large agent. The project therefore also supports agent teams and visual workflows. The workflow side supports concepts including: The UI uses XYFlow for the visual workflow editor. This lets you move from: php User - Agent - Response toward workflows such as: php - Research Agent ----- / \ Input - Classifier - Final Agent \ / - Internal Data Tool -- And because workflow runs are persisted and traced, there is something to inspect when an orchestration path doesn't behave as expected. One lesson from working with LLM applications is that manually chatting with an agent is not a sufficient testing strategy. You eventually need repeatable evaluations. Chatbot Studio includes evaluation suites so test cases can be run repeatedly instead of relying only on intuition. There is also observability around things such as: There is also a prompt optimization workflow where generated improvements can be reviewed before being accepted. I deliberately wanted that last step to stay human-controlled. An optimizer can propose a prompt change. It shouldn't silently decide that production needs a new personality at 3 AM. The project has also grown beyond browser chatbots. There is a WhatsApp channel implementation using a Node.js bridge around Baileys. It handles things including: So the architecture is increasingly becoming: php +-- Website Widget | Provider - Agent -----+-- WhatsApp | +------------- Workflows | +------------- Agent Teams The channel shouldn't define the intelligence. The agent should. Chatbot Studio isn't built as one giant application. The main pieces are: frontend/ backend/ widget/ wa-bridge/ The Studio UI is built with: Next.js 16 React 19 TypeScript Tailwind CSS Zustand XYFlow The API and agent runtime use: Python 3.12+ FastAPI Pydantic Motor MongoDB APScheduler MCP The embeddable chatbot is a separate JavaScript package built with esbuild. WhatsApp transport runs as a separate Node.js service. The default Docker setup brings together: Next.js FastAPI MongoDB WhatsApp bridge with an optional nginx SSL profile. This separation has made it much easier to reason about where functionality actually belongs. The project is designed to be self-hosted. You'll need: Node.js 20+ Python 3.12+ uv MongoDB 7 Clone the repository: git clone https://github.com/judejulius/ChatbotStudio.git cd ChatbotStudio Create your environment configuration: cp .env.example .env Install everything: npm run install:all Start MongoDB: npm run mongo Then run the development stack: npm run dev The frontend will be available on: http://localhost:3000 and FastAPI's API documentation on: http://localhost:8000/docs There's also a Docker Compose setup: docker compose up --build Make sure you replace the placeholder secrets in .env before running a real deployment. Once a platform stores model-provider credentials, MCP credentials, visitor conversations, and authentication data, security can no longer be an afterthought. The project includes mechanisms around: There is still always more security work to do in a project like this, but I've tried to make those concerns architectural rather than something added after everything else. The biggest lesson was that building an AI agent is usually not the hardest part. The hard part is everything surrounding it. A useful agent system needs answers to questions like: Where do credentials live? How are tools registered? How is knowledge retrieved? How do I test changes? How do I inspect failures? How do I embed this into another application? What happens when the AI gets stuck? Who takes over? How do I control usage? Can I change providers? Can I run it myself? Once you start answering those questions, you're no longer writing a chatbot script. You're designing infrastructure around intelligent software. That's the space I want Chatbot Studio to explore. Because I think the interesting part of AI agents right now isn't another closed chat interface. It's figuring out the infrastructure patterns that make agents useful outside demos. Things like: These are problems developers should be able to inspect, modify, argue about, and improve. Chatbot Studio is MIT licensed, so you can clone it, study it, change it, break it, rebuild parts of it, or use ideas from it in your own projects. The project has become much larger than the original chatbot-widget idea, and there are plenty of areas I want to keep improving. If you work with LLM applications, MCP, agent orchestration, RAG, frontend infrastructure, FastAPI, or self-hosted AI systems, I'd especially like feedback on the architecture. I'm interested in hearing: And if you find something broken, opening an issue is even better. The repository is here: https://github.com/judejulius/ChatbotStudio https://github.com/judejulius/ChatbotStudio If the project is useful or the architecture gives you ideas, a GitHub star is appreciated. More importantly, I'd like to see what other developers build with it.