{"slug": "i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents", "title": "I Built an Open-Source Studio for Building, Testing, and Deploying AI Agents", "summary": "A developer has released Chatbot Studio, an MIT-licensed, self-hostable open-source platform for building, testing, evaluating, and deploying AI agents as website chatbots or channel integrations such as WhatsApp. The project separates the agent (model, knowledge, tools, memory, guardrails) from its presentation layer, so a single agent remains the source of truth across multiple published chatbots, and it supports Model Context Protocol (MCP) servers, reusable knowledge bases, and multiple model providers.", "body_md": "Building an AI chatbot is easy.\n\nShipping one that has tools, knowledge, memory, observability, evaluations, human handoff, multiple model providers, workflows, and an actual interface your users can interact with is a different problem.\n\nThat gap is what led me to build **Chatbot Studio**.\n\nIt's an open-source platform for building AI agents and then publishing those agents as website chatbots or connecting them to channels such as WhatsApp.\n\nThis isn't a SaaS announcement or a sales pitch. The project is MIT licensed, self-hostable, and available on GitHub:\n\n**GitHub:** [https://github.com/judejulius/ChatbotStudio](https://github.com/judejulius/ChatbotStudio)\n\nA lot of AI projects begin with something like this:\n\n```\nresponse = client.chat.completions.create(\n    model=\"...\",\n    messages=[...]\n)\n```\n\nThen reality arrives.\n\nYou need tools.\n\nThen retrieval.\n\nThen credentials.\n\nThen streaming.\n\nThen conversation state.\n\nThen rate limits.\n\nThen evaluations.\n\nThen traces because something went wrong in production.\n\nThen a UI.\n\nThen somebody asks:\n\n\"Can we put this on the website?\"\n\nAnd someone else asks:\n\n\"Can customers talk to a human if the AI gets stuck?\"\n\nAt that point you're no longer building a prompt around an LLM.\n\nYou're building an **agent platform**.\n\nThat is the problem Chatbot Studio is trying to explore.\n\nOne architectural decision became especially important while building this.\n\nAn **agent** should not be the same object as its presentation layer.\n\nThe agent owns things such as:\n\nA published chatbot owns the things that belong to the channel:\n\nIn other words:\n\n```\nModel + Knowledge + Tools\n          |\n          v\n        Agent\n          |\n     test / evaluate\n          |\n          v\n   Published Chatbot\n          |\n   Website / Channel\n```\n\nThe chatbot isn't a duplicated agent configuration.\n\nIt's a channel sitting on top of the agent.\n\nThat means I can improve an agent's instructions, knowledge, model, or tools without recreating every chatbot using it.\n\nThe agent remains the source of truth.\n\nThe main Agent Studio lets you configure and test an agent before publishing it.\n\nCurrently the platform supports provider families including:\n\nAn agent can then be connected to knowledge bases, tools, MCP servers, memory, skills, guardrails, and other runtime controls.\n\nThe test chat runs against the saved agent configuration.\n\nThat part matters.\n\nI didn't want a playground where the test environment was secretly different from the thing that eventually gets deployed.\n\nThe goal is:\n\n``` php\nconfigure -> test -> evaluate -> publish\n```\n\nrather than:\n\n``` php\nprototype -> rewrite everything -> deploy something different\n```\n\nOne part I've been particularly interested in is **Model Context Protocol** support.\n\nChatbot Studio can connect agents to MCP servers using:\n\nThis makes external capabilities much easier to attach to an agent without baking every integration directly into the application.\n\nThe broader architecture becomes something like:\n\n```\n                 +----------------+\n                 | Knowledge Base |\n                 +-------+--------+\n                         |\n+----------+      +------v------+\n| MCP Tool +----->+             |\n+----------+      |    Agent    |\n                  |             |\n+----------+      +------+------+\n| API Tool +------------+\n+----------+             |\n                         v\n                  Conversation\n```\n\nFor me, MCP is interesting because it moves agent tooling toward a more interoperable ecosystem instead of every project inventing its own tool interface.\n\nAgents can also be connected to reusable knowledge bases.\n\nThe backend currently handles sources including:\n\nDocuments are processed into chunks and embeddings that can be retrieved during conversations.\n\nThe important idea here was making knowledge a reusable platform resource rather than stuffing documents directly into one chatbot implementation.\n\nOne knowledge base can therefore become part of a broader agent configuration.\n\nFor deployment on websites, I wanted the exported chatbot to be as framework-independent as possible.\n\nSo the primary integration is a browser-native custom element:\n\n```\n<script\n  type=\"module\"\n  src=\"https://your-domain.com/widget.esm.js?widget=wgt_...\"\n></script>\n\n<chatbot-widget></chatbot-widget>\n```\n\nThe component uses a Shadow DOM so the host website's CSS doesn't unexpectedly destroy the chatbot UI—and the chatbot doesn't leak its styles back into the host application.\n\nThe element also exposes a small JavaScript API:\n\n``` js\nconst chatbot = document.querySelector(\"chatbot-widget\");\n\nchatbot.open();\nchatbot.send(\"I need help with an order\");\nchatbot.close();\nchatbot.reset();\n```\n\nAnd it emits events such as:\n\n```\nchatbot-ready\nchatbot-error\nchatbot-handoff\n```\n\nBecause the core is a Web Component, framework integrations can stay fairly thin.\n\nChatbot Studio can generate integrations for:\n\nInstead of maintaining five completely different chatbot implementations, they all revolve around the same browser-native element.\n\nI also wanted customization to use the **real renderer**.\n\nThe editor therefore mounts the same widget implementation that gets exported.\n\nYou can customize things such as:\n\nThis avoids a problem I've seen in visual builders where the editor preview looks one way and the actual embedded component behaves differently.\n\nThe preview and the shipped widget share the same rendering path.\n\nAI shouldn't have to pretend it can solve every problem.\n\nSo Chatbot Studio also has human handoff.\n\nInstead of handing a conversation off to an undefined \"human\", conversations can be routed into queues such as:\n\n```\nGeneral Support\nTechnical Support\nSales\nBilling\n```\n\nAdministrators control which users belong to which queues.\n\nThe AI can hand the conversation to an appropriate queue, after which a person can take ownership of it.\n\nOnce handed over, the assistant stops trying to answer the conversation as though nothing happened.\n\nThis required treating handoff as application state rather than just another tool response.\n\nNot every task belongs inside one large agent.\n\nThe project therefore also supports agent teams and visual workflows.\n\nThe workflow side supports concepts including:\n\nThe UI uses XYFlow for the visual workflow editor.\n\nThis lets you move from:\n\n``` php\nUser -> Agent -> Response\n```\n\ntoward workflows such as:\n\n``` php\n                    -> Research Agent -----\n                   /                       \\\nInput -> Classifier                          -> Final Agent\n                   \\                       /\n                    -> Internal Data Tool --\n```\n\nAnd because workflow runs are persisted and traced, there is something to inspect when an orchestration path doesn't behave as expected.\n\nOne lesson from working with LLM applications is that manually chatting with an agent is not a sufficient testing strategy.\n\nYou eventually need repeatable evaluations.\n\nChatbot Studio includes evaluation suites so test cases can be run repeatedly instead of relying only on intuition.\n\nThere is also observability around things such as:\n\nThere is also a prompt optimization workflow where generated improvements can be reviewed before being accepted.\n\nI deliberately wanted that last step to stay human-controlled.\n\nAn optimizer can propose a prompt change.\n\nIt shouldn't silently decide that production needs a new personality at 3 AM.\n\nThe project has also grown beyond browser chatbots.\n\nThere is a WhatsApp channel implementation using a Node.js bridge around Baileys.\n\nIt handles things including:\n\nSo the architecture is increasingly becoming:\n\n``` php\n                      +--> Website Widget\n                      |\nProvider -> Agent -----+--> WhatsApp\n           |\n           +-------------> Workflows\n           |\n           +-------------> Agent Teams\n```\n\nThe channel shouldn't define the intelligence.\n\nThe agent should.\n\nChatbot Studio isn't built as one giant application.\n\nThe main pieces are:\n\n```\nfrontend/\nbackend/\nwidget/\nwa-bridge/\n```\n\nThe Studio UI is built with:\n\n```\nNext.js 16\nReact 19\nTypeScript\nTailwind CSS\nZustand\nXYFlow\n```\n\nThe API and agent runtime use:\n\n```\nPython 3.12+\nFastAPI\nPydantic\nMotor\nMongoDB\nAPScheduler\nMCP\n```\n\nThe embeddable chatbot is a separate JavaScript package built with esbuild.\n\nWhatsApp transport runs as a separate Node.js service.\n\nThe default Docker setup brings together:\n\n```\nNext.js\nFastAPI\nMongoDB\nWhatsApp bridge\n```\n\nwith an optional nginx SSL profile.\n\nThis separation has made it much easier to reason about where functionality actually belongs.\n\nThe project is designed to be self-hosted.\n\nYou'll need:\n\n```\nNode.js 20+\nPython 3.12+\nuv\nMongoDB 7\n```\n\nClone the repository:\n\n```\ngit clone https://github.com/judejulius/ChatbotStudio.git\ncd ChatbotStudio\n```\n\nCreate your environment configuration:\n\n```\ncp .env.example .env\n```\n\nInstall everything:\n\n```\nnpm run install:all\n```\n\nStart MongoDB:\n\n```\nnpm run mongo\n```\n\nThen run the development stack:\n\n```\nnpm run dev\n```\n\nThe frontend will be available on:\n\n```\nhttp://localhost:3000\n```\n\nand FastAPI's API documentation on:\n\n```\nhttp://localhost:8000/docs\n```\n\nThere's also a Docker Compose setup:\n\n```\ndocker compose up --build\n```\n\nMake sure you replace the placeholder secrets in `.env` before running a real deployment.\n\nOnce a platform stores model-provider credentials, MCP credentials, visitor conversations, and authentication data, security can no longer be an afterthought.\n\nThe project includes mechanisms around:\n\nThere is still always more security work to do in a project like this, but I've tried to make those concerns architectural rather than something added after everything else.\n\nThe biggest lesson was that building an AI agent is usually not the hardest part.\n\nThe hard part is everything surrounding it.\n\nA useful agent system needs answers to questions like:\n\n```\nWhere do credentials live?\n\nHow are tools registered?\n\nHow is knowledge retrieved?\n\nHow do I test changes?\n\nHow do I inspect failures?\n\nHow do I embed this into another application?\n\nWhat happens when the AI gets stuck?\n\nWho takes over?\n\nHow do I control usage?\n\nCan I change providers?\n\nCan I run it myself?\n```\n\nOnce you start answering those questions, you're no longer writing a chatbot script.\n\nYou're designing infrastructure around intelligent software.\n\nThat's the space I want Chatbot Studio to explore.\n\nBecause I think the interesting part of AI agents right now isn't another closed chat interface.\n\nIt's figuring out the infrastructure patterns that make agents useful outside demos.\n\nThings like:\n\nThese are problems developers should be able to inspect, modify, argue about, and improve.\n\nChatbot Studio is MIT licensed, so you can clone it, study it, change it, break it, rebuild parts of it, or use ideas from it in your own projects.\n\nThe project has become much larger than the original chatbot-widget idea, and there are plenty of areas I want to keep improving.\n\nIf you work with LLM applications, MCP, agent orchestration, RAG, frontend infrastructure, FastAPI, or self-hosted AI systems, I'd especially like feedback on the architecture.\n\nI'm interested in hearing:\n\nAnd if you find something broken, opening an issue is even better.\n\nThe repository is here:\n\n[https://github.com/judejulius/ChatbotStudio](https://github.com/judejulius/ChatbotStudio)\n\nIf the project is useful or the architecture gives you ideas, a GitHub star is appreciated.\n\nMore importantly, I'd like to see what other developers build with it.", "url": "https://wpnews.pro/news/i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents", "canonical_source": "https://dev.to/judejulius/i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents-1l7d", "published_at": "2026-10-01 10:38:09+00:00", "updated_at": "2026-10-01 10:44:08.502386+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "agent-protocols", "developer-tools", "large-language-models"], "entities": ["Chatbot Studio", "GitHub", "Model Context Protocol", "WhatsApp", "judejulius"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents", "markdown": "https://wpnews.pro/news/i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents.md", "text": "https://wpnews.pro/news/i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents.txt", "jsonld": "https://wpnews.pro/news/i-built-an-open-source-studio-for-building-testing-and-deploying-ai-agents.jsonld"}}