This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
AI-RAG-Bot is a local "chat with your documents" assistant: drop files into a folder, ask questions in your terminal, and get answers grounded strictly in those files. It reads .txt, .pdf, and images — and for scanned PDFs and photos of documents, it runs OCR (Tesseract + OpenCV), so even a photographed receipt becomes searchable knowledge.
Who would pay for this? Anyone who handles documents they're not allowed to paste into a cloud chatbot:
For all of them, the "easy" option today — pasting a client PDF into a hosted AI service — means shipping sensitive data to a third-party server. AI-RAG-Bot's entire value proposition is that nothing ever leaves the laptop: embeddings live in a local ChromaDB, the model runs through Ollama, and the only network call is the initial model download. And because there's no per-query API bill, the unit economics work: set it up once for a client and it costs $0/month to run, forever. Try building that margin on top of a closed API.
AI RAG Bot is a simple, locally-hosted Retrieval-Augmented Generation (RAG) system built with Python. It allows you to query a knowledge base of your own documents, including text files, PDFs, and images (using OCR). The bot processes these documents, creates vector embeddings, stores them in a local ChromaDB instance, and leverages an LLM via Ollama to generate answers grounded in the retrieved document context.
.txt, .pdf, .png, .jpg, and .jpeg files.chromadb for persistent, local storage of vector embeddings.all-MiniLM-L6-v2 model from sentence-transformers to generate embeddings.document_state.json…
Every layer of the stack is open source, and the open pieces are exactly what make the project work:
gemma3:1b — the smallest Gemma variant, about 1 GB, so it runs on modest laptops with no GPU. RAG keeps prompts short and grounded in retrieved chunks, which is exactly why a 1B model is enough here: it answers from the provided context and says "information not found" when the answer isn't in the documents.
The pipeline: scan documents/ → extract text (OCR where needed) → clean → chunk (~500 characters) → embed → store. At query time, the question is embedded too, ChromaDB returns the most similar chunks, and gemma3:1b is prompted to answer only from those chunks. That "I couldn't find that" behavior matters more than raw cleverness when someone is asking about a contract.
ollama pull anything and change one string. Different embedder? One line. No vendor decides what this tool can become.