{"slug": "a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-ai", "title": "A docs assistant a small business can run on its own laptop, built with open-weight AI", "summary": "A developer built the PumpGuard assistant, a retrieval-augmented question-and-answer system that answers from a company's own documents, cites the page used, and returns \"I could not find that in the knowledge base.\" when the answer is absent, as a Hacktoberfest Open-Source AI Challenge Week 2 submission. The build embeds 10 fictional PumpGuard pages cut into 26 chunks with BAAI/bge-small-en-v1.5 via fastembed into PostgreSQL with pgvector for hybrid vector-plus-keyword search, and generates answers with Qwen2.5 at 0.5B, 1.5B and 3B (4-bit) served by mlx-lm on a 16 GB Mac, behind an ASP.NET Core gateway and FastAPI service. The developer reports the Docker images and CI workflow were checked but never built or run, and the local model path ran once on the developer's own Mac, with tests defaulting to a stand-in writer that needs no key and costs nothing.", "body_md": "# A docs assistant a small business can run on its own laptop, built with open-weight AI\n\n*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle] hacktoberfest-week2-2026-10-12 PumpGuard assistant: a question-and-answer assistant that answers from a company's own documents (manuals, FAQs, error-code tables), cites the page it used, and says \"I could not\n\n*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle] hacktoberfest-week2-2026-10-12 PumpGuard assistant: a question-and-answer assistant that answers from a company's own documents (manuals, FAQs, error-code tables), cites the page it used, and says \"I could not find that in the knowledge base.\" when the answer isn't there. Who would pay for it? Small businesses that sell a technical product and answer the same support questions every day: \"What does error E42 mean?\", \"How do I reset the sensor?\". They have the answers in PDFs and wikis, but: their support person searches those documents by hand, again and again, and many of them can't, or don't want to, paste internal manuals into a closed AI service. The side hustle is a setup service: I install the assistant on the customer's own machine or server, load their documents, and hand over a small dashboard that shows which questions it could and couldn't answer. A one-time setup fee plus a small monthly fee to add new documents. For this build, the product and its knowledge base are fictional (the \"PumpGuard\" pump monitor, 10 pages cut into 26 chunks), so I could build and test everything without touching any real company's data. It also grew out of something bigger: I'm a developer becoming an AI engineer in 6 months, in public, and this assistant is the project my free learn-along course builds from Day 1 to the end. https://www.youtube.com/watch?v=AGSYcwMQ-1Y The video above introduces the course and the project behind it. The course itself is live: dev-to-ai-engineer.vercel.app / dev-to-ai-engineer flowchart LR user([Browser]) --> web[React chat page] web --> gateway[ASP.NET Core gateway] gateway --> ai[FastAPI AI service] ai --> db[(PostgreSQL + pgvector)] ai -.-> model{{open-weight model on your own machine}} Open-source AI sits at both ends of every answer: Finding the right pages: an open embedding model. Each chunk of the knowledge base is turned into a vector with BAAI/bge-small-en-v1.5 through fastembed, and stored in PostgreSQL with pgvector. Search is hybrid: meaning (vectors) plus keywords, so an exact error code like E42 is never missed. The embedding model is about 64 MB and runs on a plain CPU. Writing the answer: an open-weight model, locally. The AI service talks to any server that speaks the OpenAI chat API. I ran it with Qwen2.5 (0.5B, 1.5B and 3B, 4-bit) served by mlx-lm on my 16 GB Mac, and compared the sizes on the same questions. The same setting works with Ollama, llama.cpp or vLLM. Keeping it honest. The writer may answer only from the retrieved pages, every answer carries citations, a role filter keeps staff-only pages (error codes) away from customers, and an audit log records each request. Measuring instead of guessing. A fixed set of test questions checks whether the right page was found and whether the answer stayed grounded, so I can swap models and see what changed. Around it: a .NET gateway (input checks, request ids, clean errors), a React chat page with thumbs up and down, and a dashboard. What I have not run yet (I'd rather say it than hide it): the Docker images and the CI workflow were checked but never built or run, and the local model path was run once, on my own Mac. By default the tests use a stand-in writer, so they run with no key and no cost. Because for my customers, the documents are the business. Privacy: with an open embedding model and an open-weight writer, the manuals never leave the customer's machine. That single sentence is what makes a small business say yes. Cost a small business can predict: no per-token bill that grows with every support question. A laptop or a small server is enough for a 1.5B to 3B model. No lock-in: if a better open model comes out next month, I change one setting (LLM_MODEL) and re-run the test questions. A closed API can change price or behavior under me at any time. Learning in public: I could open the model, measure its memory use, and compare sizes side by side. That's how I, and the people learning with me, actually understand what a model is doing. None. This is a main-prompt entry.\n\n## Key Takeaways\n\n- •*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle] hacktoberfest-week2-2026-10-12 PumpGuard assistant: a question-and-answer assistant that answers from a company's own documents (manuals, FAQs, error-code tables), cites the page it used, and says \"I could not\n- •This story was reported by **Dev.to** , covering developments in the**dev** space.\n- •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.\n\n📖 Continue reading the full article:\n\n[Read Full Article on Dev.to →](https://dev.to/naowal/a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-weight-ai-1a4h)", "url": "https://wpnews.pro/news/a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-ai", "canonical_source": "https://ainexusdaily.vercel.app/article/2026-10-18-a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-weig", "published_at": "2026-10-18 12:28:50+00:00", "updated_at": "2026-10-18 13:18:45.023074+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-agents", "ai-infrastructure"], "entities": ["PumpGuard", "BAAI/bge-small-en-v1.5", "fastembed", "PostgreSQL", "pgvector", "Qwen2.5", "mlx-lm", "ASP.NET Core"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-ai", "markdown": "https://wpnews.pro/news/a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-ai.md", "text": "https://wpnews.pro/news/a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-ai.txt", "jsonld": "https://wpnews.pro/news/a-docs-assistant-a-small-business-can-run-on-its-own-laptop-built-with-open-ai.jsonld"}}