I recently built an "Agentic AI Knowledge Assistant" that combines "Retrieval-Augmented Generation (RAG), hybrid search, and an LLM agent" to answer questions strictly from a predefined knowledge base.
One of the main goals of this project was to address a common problem with LLM applications: the model should not answer questions using its general pretrained knowledge when the required information is not available in the knowledge base.
How it works
The system follows a retrieval-first approach:
User Question → Hybrid Retrieval → Relevant Context → AI Agent → Final Answer:
The knowledge base is divided into smaller chunks using "LangChain's RecursiveCharacterTextSplitter". Each chunk is converted into embeddings using:
"sentence-transformers/all-MiniLM-L6-v2"
For retrieval, I implemented two approaches:
The workflow looks like this:
User Query
↓
Hybrid Search
↓
FAISS || BM25
Vector || Keyword
Search || Search
↓
Relevant Knowledge Base Chunks
↓
Retrieved Context
↓
Qwen Language Model
↓
Final Answer Agentic AI Layer
The retrieval system is exposed to the agent through a custom:
knowledge_base_search() tool. The project uses smolagents CodeAgent along with the:
"Qwen/Qwen2.5-72B-Instruct" model for response generation.
The agent retrieves relevant information from the knowledge base before generating a response.
Closed-Domain Knowledge Restriction
One of the most important features of this project is the strict knowledge-base-only approach.
The assistant is instructed not to use:
If the knowledge base does not contain sufficient information, the intended response is: "The knowledge base does not contain enough content to answer this question."
This makes the system more suitable for applications where responses need to remain within a controlled information domain.
Technologies Used
What I Learned
Building this project helped me understand how different components of an AI application work together rather than treating an LLM as a standalone system.
In particular, I gained practical experience with:
Future Improvements
Some improvements I would like to implement next include:
This project was a great hands-on experience in understanding how RAG + Hybrid Search + Agentic AI + LLMs can be combined to build a more controlled AI assistant.