Request/Idea: Self-hosted LLM Linux Support RAG Recommendations/Options

A home server operator running Mistral 7B is seeking recommendations for building a self-hosted retrieval-augmented generation (RAG) database of Linux distribution support documents, aiming to diagnose issues such as UCSI power management errors without exposing usernames, machine names, or MAC addresses to third-party services. The user points to Ollama's relative ease of setup and the influx of Windows-to-Linux converts as reasons a private Linux support RAG could be a practical stepping stone, and asks whether such a tool already exists.

I recently started setting up a self-hosted LLM on my home server. I’m running Mistral 7B because my home server machine was new when Infinity War was in theaters. I’m getting my RAG stuff set up, and something which I need help with is Linux support. I don’t mean “what’s the thing I type to list the stuff in the folder?” I mean “I have a power management issue causing a UCSI error, help me diagnose the problem and resolve.” So I’m looking at starting a RAG database of documents for my distro so I can more securely and privately copy and paste command line output in there without worrying about who knows the name of my user, machine, MAC address, etc. With both the recommendations of people coming over to Linux from Windows, and then wide variety of Linux distros recommended, and the relative ease of setting up Ollama and a model, this seems like an easy stepping stone of support. Or, maybe I overestimate people’s general technical competency. Maybe both. Curious of thoughts on the subject, and maybe this is even something that’s already out there and I’ve missed it.