{"slug": "rag-performance-why-ranking-isn-t-your-real-problem", "title": "RAG Performance: Why Ranking Isn't Your Real Problem", "summary": "A developer recounts spending weeks cycling through BM25, hybrid search, cross-encoders, and multiple embedding models for RAG performance, only to find that answer quality remained flat because the retrieval stage failed to select the correct document chunk. The author argues that no reranker can promote a document absent from the candidate set, and most AI workflow bottlenecks occur at the selection layer, not the ranking layer.", "body_md": "# RAG Performance: Why Ranking Isn't Your Real Problem\n\n[RAG](/en/tags/rag/)trap. I went through a cycle of BM25, hybrid search, cross-encoders, and multiple embedding models, and while the order of results shifted every two weeks, the actual answer quality stayed flat.\n\n## The \"Different Tie\" Syndrome\n\nI was chasing a specific query: *\"Why does the gateway reject service X?\"*\n\n**BM25:** Top result was a glossary definition of \"gateway.\" (Keyword match, zero intent).**Hybrid Search:** The glossary dropped, and a Jira ticket moved up. (Symptom match, no cause).**Cross-Encoder:** The Jira ticket hit #1. (Still no answer).**New Embedding Model:** The actual architecture doc—which contained the answer—finally crawled up to #5.\n\nI felt like I was making progress because the documents were moving. In reality, I was just polishing a list of irrelevant candidates.\n\n## Selection vs. Ranking\n\nThe hard truth is that no reranker can promote a document that never made it into the candidate set. If your retrieval stage (selection) fails to pull the correct chunk, your ranking stage is just deciding which \"wrong\" answer looks the most confident.\n\nI spent way too long asking, \"How do I rank these better?\" instead of asking, \"Why are these the only candidates?\" This is where most AI workflow bottlenecks actually happen.\n\n## The Retrieval Hierarchy\n\nTo avoid this, I've started treating retrieval as a three-layer stack. If you're doing a deep dive into your LLM agent's performance, check these in order:\n\n1. **Selection:** The initial filter. What actually becomes a candidate? (This is where most failures happen).\n\n2. **Assembly:** How is the data chunked, structured, and connected? (Poor chunking kills selection).\n\n3. **Ranking:** The final sort. In what order do the candidates appear?\n\nIf your \"Selection\" layer is broken, the \"Ranking\" layer is useless. Most of us obsess over the third layer because it's where the fancy models live, but the first layer is what actually determines if the answer is even possible.\n\nStop polishing the order of documents that should never have been candidates in the first place.\n\n[Next Claude Code vs OpenAI SDK: Troubleshooting gpt-oss-120b →](/en/threads/2591/)", "url": "https://wpnews.pro/news/rag-performance-why-ranking-isn-t-your-real-problem", "canonical_source": "https://promptcube3.com/en/threads/2615/", "published_at": "2026-07-24 00:00:56+00:00", "updated_at": "2026-07-24 08:09:44.153920+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-tools", "natural-language-processing"], "entities": ["BM25", "RAG", "LLM"], "alternates": {"html": "https://wpnews.pro/news/rag-performance-why-ranking-isn-t-your-real-problem", "markdown": "https://wpnews.pro/news/rag-performance-why-ranking-isn-t-your-real-problem.md", "text": "https://wpnews.pro/news/rag-performance-why-ranking-isn-t-your-real-problem.txt", "jsonld": "https://wpnews.pro/news/rag-performance-why-ranking-isn-t-your-real-problem.jsonld"}}