{"slug": "ragit-chat-with-any-folder-of-documents-using-a-local-llm", "title": "Ragit – chat with any folder of documents using a local LLM", "summary": "Ragit, a new command-line tool, enables users to chat with any folder of documents using a local LLM via Ollama. It implements Retrieval-Augmented Generation (RAG) by indexing files, creating embeddings, and retrieving relevant chunks for answers, all while keeping data local. The tool supports .txt, .md, .pdf, and .docx files and stores indexes in ~/.ragit.", "body_md": "Local RAG CLI to chat with any folder of documents using Ollama.\n\n```\ncd ~/ragit\npython3 -m pip install -e .\n```\n\nIf your default Python is 3.14+, use Python 3.10–3.13 (recommended: 3.12) because some vector DB dependencies may not publish wheels for very new Python versions yet.\n\nMake sure Ollama is installed and running:\n\n```\nollama pull nomic-embed-text\nollama serve\n```\n\nIndex a folder:\n\n```\nragit index ./docs\n```\n\nStart chat:\n\n```\nragit chat ./docs\n```\n\nList available Ollama models:\n\n```\nragit models\n```\n\nClear an index:\n\n```\nragit clear ./docs\n```\n\n`ragit`\n\nimplements Retrieval-Augmented Generation (RAG):\n\n- It loads supported documents (\n`.txt`\n\n,`.md`\n\n,`.pdf`\n\n,`.docx`\n\n) recursively. - It splits text into overlapping chunks (about 500 words with 50-word overlap).\n- It creates embeddings using Ollama (\n`nomic-embed-text`\n\n) and stores vectors in local ChromaDB at`~/.ragit/<hash_of_path>/`\n\n. - During chat, it embeds each query, retrieves the top relevant chunks, and injects them into a prompt.\n- It streams an answer from a local Ollama chat model (prefers\n`llama3.2`\n\nif available), then shows source chunks used.\n\n- All data stays local on your machine (Ollama + Chroma local persistence).\n- Indexes are stored under\n`~/.ragit/<hash_of_path>/`\n\n. - Files that cannot be parsed are skipped with a clear error message.", "url": "https://wpnews.pro/news/ragit-chat-with-any-folder-of-documents-using-a-local-llm", "canonical_source": "https://github.com/ats4321/ragit", "published_at": "2026-07-01 06:50:52+00:00", "updated_at": "2026-07-01 07:19:50.616396+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "natural-language-processing", "developer-tools"], "entities": ["Ragit", "Ollama", "ChromaDB", "nomic-embed-text", "llama3.2"], "alternates": {"html": "https://wpnews.pro/news/ragit-chat-with-any-folder-of-documents-using-a-local-llm", "markdown": "https://wpnews.pro/news/ragit-chat-with-any-folder-of-documents-using-a-local-llm.md", "text": "https://wpnews.pro/news/ragit-chat-with-any-folder-of-documents-using-a-local-llm.txt", "jsonld": "https://wpnews.pro/news/ragit-chat-with-any-folder-of-documents-using-a-local-llm.jsonld"}}