{"slug": "ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run", "title": "AI-RAG-Bot — chat with your documents. 100% local, 100% private, $0 to run", "summary": "A developer built AI-RAG-Bot, an open-source, fully local retrieval-augmented generation assistant that lets users query their own .txt, .pdf and image files from the terminal without any data leaving the machine. The system extracts text with Tesseract and OpenCV OCR, chunks and embeds documents with all-MiniLM-L6-v2 into a local ChromaDB store, and answers queries using the roughly 1 GB gemma3:1b model via Ollama, which the developer says is sufficient because RAG keeps prompts grounded in retrieved chunks and the bot responds \"information not found\" when answers are absent. The only network call is the initial model download, making ongoing per-query cost zero.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle](https://dev.to/challenges/hacktoberfest-week2-2026-10-12)*\n\n[AI-RAG-Bot](https://github.com/sanyamkansal/AI-RAG-Bot) is a local \"chat with your documents\" assistant: drop files into a folder, ask questions in your terminal, and get answers grounded strictly in those files. It reads `.txt`, `.pdf`, and images — and for scanned PDFs and photos of documents, it runs OCR (Tesseract + OpenCV), so even a photographed receipt becomes searchable knowledge.\n\nWho would pay for this? Anyone who handles documents they're not allowed to paste into a cloud chatbot:\n\nFor all of them, the \"easy\" option today — pasting a client PDF into a hosted AI service — means shipping sensitive data to a third-party server. AI-RAG-Bot's entire value proposition is that **nothing ever leaves the laptop**: embeddings live in a local ChromaDB, the model runs through Ollama, and the only network call is the initial model download.\n\nAnd because there's no per-query API bill, the unit economics work: set it up once for a client and it costs $0/month to run, forever. Try building that margin on top of a closed API.\n\nAI RAG Bot is a simple, locally-hosted Retrieval-Augmented Generation (RAG) system built with Python. It allows you to query a knowledge base of your own documents, including text files, PDFs, and images (using OCR). The bot processes these documents, creates vector embeddings, stores them in a local ChromaDB instance, and leverages an LLM via Ollama to generate answers grounded in the retrieved document context.\n\n`.txt`, `.pdf`, `.png`, `.jpg`, and `.jpeg` files.`chromadb` for persistent, local storage of vector embeddings.`all-MiniLM-L6-v2` model from `sentence-transformers` to generate embeddings.`document_state.json`…\nEvery layer of the stack is open source, and the open pieces are exactly what make the project work:\n\n`gemma3:1b` — the smallest Gemma variant, about 1 GB, so it runs on modest laptops with no GPU. RAG keeps prompts short and grounded in retrieved chunks, which is exactly why a 1B model is enough here: it answers from the provided context and says \"information not found\" when the answer isn't in the documents.\nThe pipeline: scan `documents/` → extract text (OCR where needed) → clean → chunk (~500 characters) → embed → store. At query time, the question is embedded too, ChromaDB returns the most similar chunks, and `gemma3:1b` is prompted to answer *only* from those chunks. That \"I couldn't find that\" behavior matters more than raw cleverness when someone is asking about a contract.\n\n`ollama pull` anything and change one string. Different embedder? One line. No vendor decides what this tool can become.", "url": "https://wpnews.pro/news/ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run", "canonical_source": "https://dev.to/sanyamkansal/ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run-3d90", "published_at": "2026-10-18 19:09:30+00:00", "updated_at": "2026-10-18 19:18:57.849230+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-tools", "ai-agents"], "entities": ["AI-RAG-Bot", "Ollama", "ChromaDB", "Tesseract", "OpenCV", "sentence-transformers", "Gemma", "Hacktoberfest"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run", "markdown": "https://wpnews.pro/news/ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run.md", "text": "https://wpnews.pro/news/ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run.txt", "jsonld": "https://wpnews.pro/news/ai-rag-bot-chat-with-your-documents-100-local-100-private-0-to-run.jsonld"}}