{"slug": "rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails", "title": "RAG end to end: what each stage fixes, and what breaks when it fails", "summary": "A developer-authored lesson on Deployed Lab walks through a four-stage RAG pipeline — hybrid search, permission filtering, re-ranking, and prompt writing — showing how each stage fixes a distinct failure mode. Using a composite refund-policy example, it demonstrates that meaning-only search surfaces a marketing FAQ over the current policy and leaks a Legal-only contract page into the prompt, while mixing keyword and semantic scores or re-ranking recovers the correct 30-day answer, and only the permission filter removes the restricted page. The lesson advises debugging in pipeline order — chunking, search, permissions, re-rank, writing — and inspecting the pages that reached the prompt before blaming the model.", "body_md": "A support assistant tells a customer, \"You have 14 days to request a refund.\" The real policy says 30 days. In the same demo it quotes a contract line that only Legal may read.\n\nThe English was fluent. The assistant was simply handed the wrong pages. This is the problem RAG (retrieval-augmented generation) exists to solve, and each stage of it fixes a different failure.\n\nThe example below is a composite with made-up scores. It shows a pattern, not a benchmark.\n\nAn AI that has never seen your files will guess, and the guess sounds confident. RAG lets it look up the right pages just before it answers. Looking things up is cheap and always current. Retraining goes stale the day a policy changes.\n\nYour documents are filed once, in advance, as chunks, each stamped with who may read it. Then every question passes through these stages.\n\n**1. Search.** Hybrid search matches by meaning and by exact words, then mixes the scores.\n\n**2. Permission check.** Pages the asker may not read are dropped, using the access list stored with each chunk.\n\n**3. Re-rank.** A slower, more careful reader re-sorts the pages that search found.\n\n**4. Writing.** The rules, top pages and question go into one prompt, and the AI answers.\n\nThe question: \"How long do Enterprise customers have to request a refund?\" Four pages, made-up scores from 0 to 1:\n\n| Page | Meaning | Keyword | Careful reader | \n|---|---|---|---|\n| Refund policy v3 (current) | 0.74 | 0.95 | 0.97 | \n| Refund policy v1 (old) | 0.82 | 0.55 | 0.30 | \n| Marketing FAQ | 0.88 | 0.35 | 0.20 | \n| Legal-only contract page | 0.79 | 0.60 | 0.55 | \n\nOnly the top 3 reach the AI. With meaning alone, the FAQ wins and the current policy is cut. The answer is vague, and the Legal-only page is in the prompt.\n\nNow mix half meaning and half keyword:\n\nThe current policy rises to first place and the answer becomes 30 days. The re-ranker reaches the same result from a different stage: it scores the current policy 0.97 against 0.30 for the old one.\n\nOnly the permission filter removes the Legal-only page.\n\nCheck in order: shelving (chunk size), search, permission check, re-rank, writing. An early fault hides later ones. Read the pages that went into the prompt first, because a wrong number could be a stale page or the AI inventing it.\n\nThe chapter has an interactive lab where you switch hybrid search, the re-ranker and the permission filter on and off, and watch the answer change. It takes about 12 minutes and is free: [RAG end to end](https://deployedlab.dev/lessons/09-rag-end-to-end). It is one of the free chapters of [Deployed Lab](https://deployedlab.dev), a course on AI engineering and the forward deployed engineer role.", "url": "https://wpnews.pro/news/rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails", "canonical_source": "https://dev.to/deployedlabs/rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails-5fn3", "published_at": "2026-11-20 21:05:00+00:00", "updated_at": "2026-11-20 21:15:00.579192+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-agents"], "entities": ["Deployed Lab"], "also_reported_by": [], "takeaway": "Hybrid search mixing half semantic and half keyword scores promoted the current refund policy to first place and produced the correct 30-day answer, but only the per-chunk permission filter removed the Legal-only page from the prompt.", "recommendation": {"advice": "Combine semantic and keyword scores and add a re-ranker to surface current documents, but enforce per-chunk access lists as a separate filter step, since retrieval alone will leak restricted pages into the prompt.", "playbook": "rag-retrieval", "playbook_url": "https://wpnews.pro/playbooks/rag-retrieval"}, "alternates": {"html": "https://wpnews.pro/news/rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails", "markdown": "https://wpnews.pro/news/rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails.md", "text": "https://wpnews.pro/news/rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails.txt", "jsonld": "https://wpnews.pro/news/rag-end-to-end-what-each-stage-fixes-and-what-breaks-when-it-fails.jsonld"}}