Anatomy of a Full RAG Application: Every Concept, One Self-Hosted Stack

A developer built myRAG, a fully self-hosted RAG stack combining FastAPI, React, Qdrant, PostgreSQL, and Neo4j. The system uses hybrid search with dense and sparse embeddings, cross-encoder reranking, and knowledge graph triples to improve retrieval accuracy for complex queries. The project demonstrates a complete pipeline from document ingestion to streaming LLM responses.

"Chat with your documents" sounds simple. Then you build it, and you discover a good RAG system is really eight systems wearing a trench coat. I recently finished myRAG — a fully self-hosted RAG stack: FastAPI backend, React frontend, and three storage engines Qdrant, PostgreSQL, Neo4j , all orchestrated with Docker Compose. This post walks through every stage of the pipeline and the concept behind it, with real code from the project. For a link to the codebase, scroll to bottom Document ─► Docling ─► Chunker ─┬─► Dense embed OpenRouter ──┐ ├─► Sparse BM25 fastembed ───┼─► Qdrant named vectors └─► LLM triple extraction ─────┼─► Neo4j knowledge graph └─► PostgreSQL metadata, doc uuid Question ─► hybrid search RRF ─► rerank ─► + graph facts ─► + memory ─► token budget ─► LLM ─► SSE stream Six containers: the app, the React/nginx frontend, docling-serve , Qdrant, Postgres, and Neo4j. Everything starts with converting PDFs/DOCX/HTML into text an LLM can use. I run Docling https://github.com/docling-project/docling as a separate service that returns clean Markdown , preserving headings and tables. Why Markdown? Because structure survives. A table flattened into a character soup is unsearchable no matter how good your embedding model is. Garbage in, garbage out — this stage silently determines your ceiling. Chunks are the retrieval unit. Too large and the embedding becomes a blurry average of many topics; too small and the chunk loses the context needed to be understood alone. chunking: strategy: recursive chunk size: 512 chunk overlap: 64 Recursive splitting respects paragraph and sentence boundaries, and the overlap is essential: without it, answers that straddle a chunk boundary fall into the gap and are never retrieved whole. Most tutorials embed once and call it done. I index every chunk two ways : fastembed , no API cost — lexical similarity. Exact part numbers, names, acronyms — the things embedding models fumble.Qdrant stores both on the same point as named vectors , with BM25's IDF computed server-side: self.client.create collection collection name=..., vectors config={"dense": VectorParams size=4096, distance=Distance.COSINE }, sparse vectors config={"bm25": models.SparseVectorParams modifier=models.Modifier.IDF }, One subtlety: BM25 weights documents and queries differently, so ingestion uses embed term weighting while queries use query embed term presence . At query time, both searches run and Qdrant fuses them with RRF — which merges ranked lists instead of trying to normalize incomparable score scales: result = self.client.query points collection name=..., prefetch= models.Prefetch query=dense vector, using="dense", limit=20 , models.Prefetch query=sparse query, using="bm25", limit=20 , , query=models.FusionQuery fusion=models.Fusion.RRF , limit=10, RRF scores each document by Σ 1/ k + rank across lists. A chunk ranked highly by either method surfaces; one ranked highly by both wins. No score calibration, no tuning — and it works disturbingly well. Vector search compares a query against chunks that were embedded without knowing the question . A cross-encoder reranker Cohere rerank, via OpenRouter reads query and chunk together and produces a much sharper relevance score. The pattern is a funnel: hybrid-retrieve 10 candidates cheaply, rerank down to the best 5. It's the same two-stage architecture search engines have used for decades — recall first, precision second. Pure vector RAG struggles with relational questions — "Who reports to the person who founded X?" spans facts that live in different chunks. So during ingestion, an LLM extracts subject, relation, object triples from every chunk into Neo4j: You are a knowledge-graph extraction engine. From the text below, extract factual relationships as a JSON array of triples... Only extract relationships explicitly stated in the text. Return ONLY the JSON array, no prose. At query time the flow is: extract entities from the question → match them against graph nodes via Neo4j's fulltext index → pull their 1-hop neighborhood → inject the triples into the prompt as structured facts: php Knowledge graph facts: - Acme Corp --acquired-- Widget Inc - Widget Inc --founded by-- Jane Doe Call it GraphRAG-lite: the vector store answers "what's relevant," the graph answers "how things relate." Every relationship is tagged with doc uuid , so deleting a document prunes its facts and any orphaned entities cleanly. Multi-turn chat needs history, but the context window is finite. Two mechanisms: Rolling summarization — after N turns, older exchanges are compressed into a running summary by a small LLM. Long conversations cost a paragraph, not pages. Token budgeting — before generation, the prompt is assembled against a hard cap with explicit priorities: Priority always kept : system prompt, summary, graph facts, the question. Then newest history, then chunks best-first; lowest-ranked chunks drop first. while history msgs and fixed + est history msgs budget: history msgs.pop 0 oldest history goes first for chunk in chunks: rerank order: best first if used + est chunk avail and len kept = min chunks: break kept.append chunk The mental model: every token of history you keep is a token of evidence you can't include. Making that trade-off explicit — instead of letting the API truncate arbitrarily — noticeably improves long conversations. Ingestion is embarrassingly parallel — embedding API calls, local BM25 encoding, and per-chunk graph extraction are all independent I/O. A single shared thread pool handles the fan-out: python executor.py — one process-wide pool, order-preserving map def map parallel fn, items : items = list items if len items <= 1: return fn i for i in items return list get pool .map fn, items Three applications: 1. Dense embedding: sub-batches of 32 texts, requests in flight concurrently batches = texts i:i+32 for i in range 0, len texts , 32 vectors = v for batch in map parallel self. embed request, batches for v in batch 2. Sparse BM25 runs concurrently with the dense API round-trips sparse future = get pool .submit self.sparse embedder.embed batch, texts vectors = self.embedder.embed batch texts sparse vectors = sparse future.result 3. Graph extraction: one LLM call per chunk, previously sequential — now fanned out futures = get pool .submit self.graph extractor.extract, t for t in texts for i, future in enumerate as completed futures : ... upsert triples + yield progress events as each completes At query time, the graph fact lookup is submitted before retrieval starts, so it overlaps with retrieval + reranking instead of running after them. Graph extraction was the slowest ingestion phase by far — parallelizing it is the difference between 90 seconds and 15 for a mid-sized document. No async rewrite needed; threads are plenty for I/O-bound work. doc uuid ties each document across Postgres primary key , Qdrant point payload , and Neo4j relationship property . Deletes cascade through all three stores. parsing → chunking → embedding → storing → graph → done live. One gotcha: EventSource can't send headers, so the client streams via fetch + ReadableStream to pass the auth token. /api/ endpoint requires a bearer token, checked with secrets.compare digest constant-time . The React build bakes the key in at build time from an env var — no login screen; docker-compose sources frontend and backend from the same .env so they can't drift. config.yaml for every tunable, Pydantic Settings overlays secrets from .env , zero hardcoded values.RAG isn't one technique. It's a pipeline of small, well-understood ideas: Parse cleanly → chunk thoughtfully → index twice → fuse ranks → rerank deeply → add structure with a graph → budget your tokens → parallelize the waits. None of these steps is hard alone. The engineering is in making them agree with each other — sharing one UUID, one config, one thread pool, and one honest token budget. Access the full codebase here: https://github.com/sumannath/myRAG https://github.com/sumannath/myRAG Questions about any layer? The hybrid search and knowledge-graph stages delivered the biggest quality jumps for me, and I'm happy to go deeper on either in the comments.