# AI-RAG-Bot — chat with your documents. 100% local, 100% private, $0 to run > Source: > Published: 2026-10-18 19:09:30+00:00 *This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle](https://dev.to/challenges/hacktoberfest-week2-2026-10-12)* [AI-RAG-Bot](https://github.com/sanyamkansal/AI-RAG-Bot) is a local "chat with your documents" assistant: drop files into a folder, ask questions in your terminal, and get answers grounded strictly in those files. It reads `.txt`, `.pdf`, and images — and for scanned PDFs and photos of documents, it runs OCR (Tesseract + OpenCV), so even a photographed receipt becomes searchable knowledge. Who would pay for this? Anyone who handles documents they're not allowed to paste into a cloud chatbot: For all of them, the "easy" option today — pasting a client PDF into a hosted AI service — means shipping sensitive data to a third-party server. AI-RAG-Bot's entire value proposition is that **nothing ever leaves the laptop**: embeddings live in a local ChromaDB, the model runs through Ollama, and the only network call is the initial model download. And because there's no per-query API bill, the unit economics work: set it up once for a client and it costs $0/month to run, forever. Try building that margin on top of a closed API. AI RAG Bot is a simple, locally-hosted Retrieval-Augmented Generation (RAG) system built with Python. It allows you to query a knowledge base of your own documents, including text files, PDFs, and images (using OCR). The bot processes these documents, creates vector embeddings, stores them in a local ChromaDB instance, and leverages an LLM via Ollama to generate answers grounded in the retrieved document context. `.txt`, `.pdf`, `.png`, `.jpg`, and `.jpeg` files.`chromadb` for persistent, local storage of vector embeddings.`all-MiniLM-L6-v2` model from `sentence-transformers` to generate embeddings.`document_state.json`… Every layer of the stack is open source, and the open pieces are exactly what make the project work: `gemma3:1b` — the smallest Gemma variant, about 1 GB, so it runs on modest laptops with no GPU. RAG keeps prompts short and grounded in retrieved chunks, which is exactly why a 1B model is enough here: it answers from the provided context and says "information not found" when the answer isn't in the documents. The pipeline: scan `documents/` → extract text (OCR where needed) → clean → chunk (~500 characters) → embed → store. At query time, the question is embedded too, ChromaDB returns the most similar chunks, and `gemma3:1b` is prompted to answer *only* from those chunks. That "I couldn't find that" behavior matters more than raw cleverness when someone is asking about a contract. `ollama pull` anything and change one string. Different embedder? One line. No vendor decides what this tool can become.