# Anatomy of a Full RAG Application: Every Concept, One Self-Hosted Stack > Source: > Published: 2026-07-12 13:35:29+00:00 "Chat with your documents" sounds simple. Then you build it, and you discover a good RAG system is really eight systems wearing a trench coat. I recently finished **myRAG** — a fully self-hosted RAG stack: FastAPI backend, React frontend, and three storage engines (Qdrant, PostgreSQL, Neo4j), all orchestrated with Docker Compose. This post walks through every stage of the pipeline and the *concept* behind it, with real code from the project. For a link to the codebase, scroll to bottom! ``` Document ─► Docling ─► Chunker ─┬─► Dense embed (OpenRouter) ──┐ ├─► Sparse BM25 (fastembed) ───┼─► Qdrant (named vectors) └─► LLM triple extraction ─────┼─► Neo4j (knowledge graph) └─► PostgreSQL (metadata, doc_uuid) Question ─► hybrid search (RRF) ─► rerank ─► + graph facts ─► + memory ─► token budget ─► LLM ─► SSE stream ``` Six containers: the app, the React/nginx frontend, `docling-serve` , Qdrant, Postgres, and Neo4j. Everything starts with converting PDFs/DOCX/HTML into text an LLM can use. I run [Docling](https://github.com/docling-project/docling) as a separate service that returns clean **Markdown**, preserving headings and tables. Why Markdown? Because structure survives. A table flattened into a character soup is unsearchable no matter how good your embedding model is. Garbage in, garbage out — this stage silently determines your ceiling. Chunks are the retrieval unit. Too large and the embedding becomes a blurry average of many topics; too small and the chunk loses the context needed to be understood alone. ``` chunking: strategy: recursive chunk_size: 512 chunk_overlap: 64 ``` Recursive splitting respects paragraph and sentence boundaries, and the **overlap** is essential: without it, answers that straddle a chunk boundary fall into the gap and are never retrieved whole. Most tutorials embed once and call it done. I index every chunk **two ways**: `fastembed` , no API cost) — lexical similarity. Exact part numbers, names, acronyms — the things embedding models fumble.Qdrant stores both on the same point as *named vectors*, with BM25's IDF computed server-side: ``` self.client.create_collection( collection_name=..., vectors_config={"dense": VectorParams(size=4096, distance=Distance.COSINE)}, sparse_vectors_config={"bm25": models.SparseVectorParams(modifier=models.Modifier.IDF)}, ) ``` One subtlety: BM25 weights documents and queries differently, so ingestion uses `embed()` (term weighting) while queries use `query_embed()` (term presence). At query time, both searches run and Qdrant fuses them with **RRF** — which merges *ranked lists* instead of trying to normalize incomparable score scales: ``` result = self.client.query_points( collection_name=..., prefetch=[ models.Prefetch(query=dense_vector, using="dense", limit=20), models.Prefetch(query=sparse_query, using="bm25", limit=20), ], query=models.FusionQuery(fusion=models.Fusion.RRF), limit=10, ) ``` RRF scores each document by `Σ 1/(k + rank)` across lists. A chunk ranked highly by *either* method surfaces; one ranked highly by *both* wins. No score calibration, no tuning — and it works disturbingly well. Vector search compares a query against chunks that were embedded *without knowing the question*. A **cross-encoder reranker** (Cohere rerank, via OpenRouter) reads query and chunk *together* and produces a much sharper relevance score. The pattern is a funnel: hybrid-retrieve 10 candidates cheaply, rerank down to the best 5. It's the same two-stage architecture search engines have used for decades — recall first, precision second. Pure vector RAG struggles with relational questions — *"Who reports to the person who founded X?"* spans facts that live in different chunks. So during ingestion, an LLM extracts **(subject, relation, object)** triples from every chunk into Neo4j: ``` You are a knowledge-graph extraction engine. From the text below, extract factual relationships as a JSON array of triples... Only extract relationships explicitly stated in the text. Return ONLY the JSON array, no prose. ``` At query time the flow is: extract entities from the question → match them against graph nodes via Neo4j's fulltext index → pull their 1-hop neighborhood → inject the triples into the prompt as structured facts: ``` php Knowledge graph facts: - Acme Corp --acquired--> Widget Inc - Widget Inc --founded_by--> Jane Doe ``` Call it GraphRAG-lite: the vector store answers *"what's relevant,"* the graph answers *"how things relate."* Every relationship is tagged with `doc_uuid` , so deleting a document prunes its facts (and any orphaned entities) cleanly. Multi-turn chat needs history, but the context window is finite. Two mechanisms: **Rolling summarization** — after N turns, older exchanges are compressed into a running summary by a small LLM. Long conversations cost a paragraph, not pages. **Token budgeting** — before generation, the prompt is assembled against a hard cap with explicit priorities: ``` # Priority (always kept): system prompt, summary, graph facts, the question. # Then newest history, then chunks best-first; lowest-ranked chunks drop first. while history_msgs and fixed + est(history_msgs) > budget: history_msgs.pop(0) # oldest history goes first for chunk in chunks: # rerank order: best first if used + est(chunk) > avail and len(kept) >= min_chunks: break kept.append(chunk) ``` The mental model: every token of history you keep is a token of evidence you can't include. Making that trade-off explicit — instead of letting the API truncate arbitrarily — noticeably improves long conversations. Ingestion is embarrassingly parallel — embedding API calls, local BM25 encoding, and per-chunk graph extraction are all independent I/O. A single shared thread pool handles the fan-out: ``` python # executor.py — one process-wide pool, order-preserving map def map_parallel(fn, items): items = list(items) if len(items) <= 1: return [fn(i) for i in items] return list(get_pool().map(fn, items)) ``` Three applications: ``` # 1. Dense embedding: sub-batches of 32 texts, requests in flight concurrently batches = [texts[i:i+32] for i in range(0, len(texts), 32)] vectors = [v for batch in map_parallel(self._embed_request, batches) for v in batch] # 2. Sparse BM25 runs concurrently with the dense API round-trips sparse_future = get_pool().submit(self.sparse_embedder.embed_batch, texts) vectors = self.embedder.embed_batch(texts) sparse_vectors = sparse_future.result() # 3. Graph extraction: one LLM call per chunk, previously sequential — now fanned out futures = [get_pool().submit(self.graph_extractor.extract, t) for t in texts] for i, future in enumerate(as_completed(futures)): ... # upsert triples + yield progress events as each completes ``` At query time, the graph fact lookup is submitted before retrieval starts, so it overlaps with retrieval + reranking instead of running after them. Graph extraction was the slowest ingestion phase by far — parallelizing it is the difference between 90 seconds and 15 for a mid-sized document. No async rewrite needed; threads are plenty for I/O-bound work. `doc_uuid` ties each document across Postgres (primary key), Qdrant (point payload), and Neo4j (relationship property). Deletes cascade through all three stores.`parsing → chunking → embedding → storing → graph → done` live. One gotcha: `EventSource` can't send headers, so the client streams via `fetch` + `ReadableStream` to pass the auth token.`/api/*` endpoint requires a bearer token, checked with `secrets.compare_digest` (constant-time). The React build bakes the key in at build time from an env var — no login screen; docker-compose sources frontend and backend from the same `.env` so they can't drift.`config.yaml` for every tunable, Pydantic Settings overlays secrets from `.env` , zero hardcoded values.RAG isn't one technique. It's a pipeline of small, well-understood ideas: Parse cleanly → chunk thoughtfully → index twice → fuse ranks → rerank deeply → add structure with a graph → budget your tokens → parallelize the waits. None of these steps is hard alone. The engineering is in making them agree with each other — sharing one UUID, one config, one thread pool, and one honest token budget. Access the full codebase here: [https://github.com/sumannath/myRAG](https://github.com/sumannath/myRAG) Questions about any layer? The hybrid search and knowledge-graph stages delivered the biggest quality jumps for me, and I'm happy to go deeper on either in the comments.