RAGexamples you see online are just wrappers around a paid API, which makes it impossible to actually benchmark cost or latency on your own hardware. I found a hands-on guide in the form of a workshop (happening August 29) that actually tackles the deployment of entirely open-source stacks. It's led by Ben Auffarth, who specializes in this kind of AI workflow optimization.
The technical focus here is on the parts of the pipeline that usually break when you move from a demo to a real-world application. Instead of just throwing documents into a vector store and hoping for the best, this covers a more sophisticated hybrid retrieval setup. Combining vector search with keyword search is usually the only way to stop the LLM from hallucinating when the user asks for a specific term that doesn't have a strong semantic embedding.
They're also diving into reranking. If you've built a RAG pipeline, you know that the top-k results from a vector search are often noisy. Adding a reranker is a practical tutorial in improving precision by filtering those chunks before they ever hit the context window.
The Technical Stack and Evaluation #
What actually interests me is the inclusion of RAGAS for evaluation. Too many people "vibe check" their RAG systems—meaning they ask three questions, get okay answers, and assume the system works. Using RAGAS allows for a deep dive into faithfulness and relevancy, turning qualitative guesses into quantitative metrics.
The workshop also covers:
Guardrails: Implementing constraints during the design phase rather than patching them in later.Benchmarking: Real-world performance data on open-model deployments so you know exactly what hardware you need.Hybrid Search: Moving beyond simple cosine similarity to include keyword-based retrieval.
For anyone wanting to build from scratch without being locked into a specific provider's ecosystem, this looks like a solid roadmap. It treats RAG as an engineering problem rather than just a prompt engineering trick.
The event details are here:
https://www.eventbrite.co.uk/e/the-genai-build-lab-build-production-ready-rag-on-a-budget-tickets-1994016271345?aff=rml
I'm curious if anyone here has tried combining RAGAS with open-source rerankers like BGE-Reranker—does the latency hit usually justify the jump in accuracy for your use cases?
Next Can a sole author skip the ICONIP 2026 in-person presentation? →