{"slug": "building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag", "title": "Building a Closed-Domain Agentic AI Knowledge Assistant with Hybrid RAG", "summary": "A developer built an 'Agentic AI Knowledge Assistant' that combines hybrid retrieval-augmented generation (RAG) with an LLM agent to answer questions strictly from a predefined knowledge base. The system uses LangChain for text splitting, sentence-transformers for embeddings, and FAISS/BM25 for hybrid search, with a smolagents CodeAgent and Qwen2.5-72B-Instruct for response generation. The assistant is designed to refuse answering when the knowledge base lacks sufficient information, ensuring closed-domain responses.", "body_md": "I recently built an **\"Agentic AI Knowledge Assistant\"** that combines \"Retrieval-Augmented Generation (RAG), hybrid search, and an LLM agent\" to answer questions strictly from a predefined knowledge base.\n\nOne of the main goals of this project was to address a common problem with LLM applications: **the model should not answer questions using its general pretrained knowledge when the required information is not available in the knowledge base.**\n\n**How it works**\n\nThe system follows a retrieval-first approach:\n\nUser Question → Hybrid Retrieval → Relevant Context → AI Agent → Final Answer:\n\nThe knowledge base is divided into smaller chunks using \"LangChain's RecursiveCharacterTextSplitter\". Each chunk is converted into embeddings using:\n\n\"sentence-transformers/all-MiniLM-L6-v2\"\n\nFor retrieval, I implemented two approaches:\n\nThe workflow looks like this:\n\nUser Query\n\n↓\n\nHybrid Search\n\n↓\n\nFAISS || BM25\n\nVector || Keyword\n\nSearch || Search\n\n↓\n\nRelevant Knowledge Base Chunks\n\n↓\n\nRetrieved Context\n\n↓\n\nQwen Language Model\n\n↓\n\nFinal Answer\n\n**Agentic AI Layer**\n\nThe retrieval system is exposed to the agent through a custom:\n\nknowledge_base_search() tool.\n\nThe project uses **smolagents CodeAgent** along with the:\n\n\"Qwen/Qwen2.5-72B-Instruct\" model for response generation.\n\nThe agent retrieves relevant information from the knowledge base before generating a response.\n\n**Closed-Domain Knowledge Restriction**\n\nOne of the most important features of this project is the strict knowledge-base-only approach.\n\nThe assistant is instructed not to use:\n\nIf the knowledge base does not contain sufficient information, the intended response is:\n\n\"The knowledge base does not contain enough content to answer this question.\"\n\nThis makes the system more suitable for applications where responses need to remain within a controlled information domain.\n\nTechnologies Used\n\n**What I Learned**\n\nBuilding this project helped me understand how different components of an AI application work together rather than treating an LLM as a standalone system.\n\nIn particular, I gained practical experience with:\n\n**Future Improvements**\n\nSome improvements I would like to implement next include:\n\nThis project was a great hands-on experience in understanding how **RAG + Hybrid Search + Agentic AI + LLMs** can be combined to build a more controlled AI assistant.", "url": "https://wpnews.pro/news/building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag", "canonical_source": "https://dev.to/sanjay_sajukumar_04/building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag-2416", "published_at": "2026-08-30 04:56:47+00:00", "updated_at": "2026-08-30 05:22:30.731942+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-agents", "developer-tools"], "entities": ["LangChain", "sentence-transformers/all-MiniLM-L6-v2", "FAISS", "BM25", "Qwen/Qwen2.5-72B-Instruct", "smolagents"], "alternates": {"html": "https://wpnews.pro/news/building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag", "markdown": "https://wpnews.pro/news/building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag.md", "text": "https://wpnews.pro/news/building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag.txt", "jsonld": "https://wpnews.pro/news/building-a-closed-domain-agentic-ai-knowledge-assistant-with-hybrid-rag.jsonld"}}