{"slug": "how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a", "title": "How would you build an AI assistant that actually becomes an expert on a software platform?", "summary": "A developer is seeking advice on building a local AI assistant that becomes an expert on a complex software platform, using small models under 3B parameters without GPU inference. The assistant should answer questions, explain concepts, and guide users using documentation, videos, and scripts. The developer is evaluating RAG, fine-tuning, and agent-based architectures.", "body_md": "Hi everyone, I’m trying to figure out the best way to build an AI assistant that can become a **real expert on a complex software platform**.\n\nThe goal isn’t just to build a chatbot that answers documentation questions. I want it to behave more like an experienced product expert. It should be able to answer questions, explain concepts, help users build workflows, diagnose errors, recommend best practices, explain configuration options, and generally guide users through the software.\n\nThe software has a few hundred pages of documentation covering features, automation actions, parameters, examples, troubleshooting guides, architecture, and best practices. There are also tutorial videos, knowledge base articles, and a large collection of existing automation scripts that are actively used. Ideally, the assistant should be able to leverage all of this knowledge to provide accurate, reliable, and well-grounded answers.\n\nOne important **constraint** is that I’d like the final solution to **run locally**, preferably on modest hardware **without requiring a GPU for inference**. Models under **3B** parameter range seem realistic from a deployment perspective, but I’m not sure whether models that size can actually become experts on a complex software domain.\n\nI’ve read about several possible approaches: Continued Pretraining (CPT) Supervised Fine-Tuning (SFT) RAG Agent-based systems with LangGraph\n\nWhat I’m struggling with is understanding what the overall architecture should actually look like. If you’ve built something similar I’d really appreciate hearing about your experience or If you know of any **similar projects**, I’d love to check them out.\n\nSome questions I’m wondering about: What **architecture** ended up working best? Can a relatively **small local model realistically become an expert** on a software platform, or is that expecting too much? Is RAG enough for this kind of problem, or would you still invest in CPT or fine-tuning? Would you use a single model, multiple specialized agents, or a different architecture altogether? If you were starting this project from scratch today, how would you design it? Thanks in advance !", "url": "https://wpnews.pro/news/how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a", "canonical_source": "https://discuss.huggingface.co/t/how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a-software-platform/177855#post_1", "published_at": "2026-07-15 14:28:10+00:00", "updated_at": "2026-07-15 14:30:33.821726+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "developer-tools"], "entities": ["LangGraph"], "alternates": {"html": "https://wpnews.pro/news/how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a", "markdown": "https://wpnews.pro/news/how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a.md", "text": "https://wpnews.pro/news/how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a.txt", "jsonld": "https://wpnews.pro/news/how-would-you-build-an-ai-assistant-that-actually-becomes-an-expert-on-a.jsonld"}}