One inference box (DGX Spark) and multiple use cases... A developer built an experimental RAG system, RAG-LCC, and spun out a separate GitHub repo, harinezumigel-llm-stack, to manage the multiple LLMs it relies on. The project uses separate models for prompt guarding and judging rather than a single inference model, and maintains two configuration sets for switching between use cases. The developer notes that running local coding assistance on a single DGX Spark box competes with the RAG tests for compute. While developing my RAG-LCC https://github.com/HarinezumIgel/RAG-LCC experimental RAG, I had multiple use cases: RAG-LCC uses LLMs for: Although some of the tasks can be handled by the same LMM it is good practice to have other models then the inference one for prompt guard and judgement. I use 2 sets of configurations, which I want to switch as convenient as possible. Coding assistance: I tried different models, which would be a separate discussion. But when I use local coding assistance, I cannot run the RAG tests because my Spark DGX can't provide it. Building a RAG system taught me that managing the LLMs I use is an important part of my development flow. What started as some lines of python code has become a GitHub repo on its own. If you want to have a look: harinezumigel-llm-stack https://github.com/HarinezumIgel/harinezumigel-llm-stack Feedback and ideas are welcome.