Hexagonal Architecture for Provider-Agnostic RAG Pipelines

A developer built a provider-agnostic RAG pipeline using hexagonal architecture, decoupling the application from specific LLMs, vector databases, and document-processing frameworks through lazy imports and abstract interfaces. The demo stack combines LangChain, llama.cpp, Mistral.ai, Qwen embeddings, PostgreSQL with pgvector, and Telegraf/Prometheus observability on Docker, with the full implementation published on GitHub as yoga1290/rag.

Demo stack: LangChain, Llama.cpp, Mistral.ai, Qwen Embeddings, PostgreSQL, Telegraf, Prometheus on Docker. I recently came across Machine Learning Mastery's guide https://machinelearningmastery.com/building-a-rag-pipeline-with-llama-cpp-in-python/ on Building a RAG Pipeline with llama.cpp and decided to try it myself. One thing quickly became apparent: some of the APIs and methods used in the example had already changed or become deprecated. That led me to a bigger question: How do you design a RAG system that can evolve as the underlying technologies change? Instead of tightly coupling the application to a specific LLM, vector database, or document-processing framework, I experimented with a more modular architecture. This way I get a more resilient architecture, for instance if I need a transition between local on-premise to fully on cloud or just hybrid; ability to switch to different database like PostgreSQL instead of ChromaDB, due to the data integration and ACID compliance A few principles became particularly important: python def importOnCall .. : from.. import .. Lazy import return ResultModel .. tied to Abstracts & Models vectorstore: VectorStore = PGVectorStore ... vectorstore: VectorStore = OtherVectorDBService .. 🛠️ Separation of responsibilities Each layer has a focused responsibility: 📦️ Observability, Containerization & Resource/Network monitoring , Confiurations are pass in form of Environment Variables , see sample.env https://github.com/yoga1290/rag/raw/master/sample.env . Telegraf is used to monitor the resource consumsion and network traffic , and project metrics to Prometheus . See the 👁️ Observability section. The complete implementation is available on GitHub: yoga1290/rag