My RAG retrieval quality is tanking as my dataset grows

A developer reports that retrieval precision in their RAG pipeline degrades as the dataset grows, using SentenceTransformer's all-MiniLM-L6-v2 and FAISS IndexFlatIP with L2-normalized vectors. They are considering switching to HNSW or IVF indexes, adding a Cross-Encoder reranker, and improving metadata filtering to address the issue.

My RAG retrieval quality is tanking as my dataset grows I've been building a Since I'm using I'm noticing that cosine similarity often retrieves chunks that are semantically related but logically useless for the actual answer. I'm thinking about adding a reranker stage—pulling 50 candidates via FAISS and then narrowing it down to the top 5 using a Cross-Encoder. Does this actually solve the precision issue, or is it just adding latency to the pipeline? I'm using Right now, I store metadata in a separate Python list RAG /en/tags/rag/ pipeline using SentenceTransformer and FAISS. On a tiny test set, everything looks great, but as I scale up, I'm hitting a wall with retrieval precision. I'm currently using IndexFlatIP with L2-normalized vectors to simulate cosine similarity. It's basically a brute-force search, which is fine for a few hundred documents, but I'm worried about the latency and accuracy trade-offs as I move toward a real-world deployment.Here is the core of my current implementation: python class VectorStore: def init self : self.model = SentenceTransformer EMBEDDING MODEL self.index = None self.chunks = def build self, chunks : self.chunks = chunks texts = chunk "text" for chunk in chunks embeddings = self.model.encode texts, convert to numpy=True, show progress bar=True .astype "float32" Normalize embeddings so inner product = cosine similarity faiss.normalize L2 embeddings dimension = embeddings.shape 1 self.index = faiss.IndexFlatIP dimension self.index.add embeddings def search self, query, k=5 : query embedding = self.model.encode query , convert to numpy=True .astype "float32" faiss.normalize L2 query embedding scores, indices = self.index.search query embedding, k results = for score, index in zip scores 0 , indices 0 : if index == -1: continue results.append { "text": self.chunks index "text" , "chunk id": self.chunks index "chunk id" , "score": float score } return results I've realized that just having "similar" vectors isn't enough for a production-grade AI workflow. I have a few specific technical hurdles I'm trying to clear: The Indexing Bottleneck Since I'm using IndexFlatIP , every query scans the entire index. I'm considering switching to IndexHNSWFlat or IndexIVFFlat to speed things up, but I'm unsure about the recall hit. In a practical tutorial for RAG, you often see HNSW recommended for speed, but is the memory overhead worth it compared to something like IndexIVFPQ for larger datasets? The "Vector Search is Not Enough" Problem I'm noticing that cosine similarity often retrieves chunks that are semantically related but logically useless for the actual answer. I'm thinking about adding a reranker stage—pulling 50 candidates via FAISS and then narrowing it down to the top 5 using a Cross-Encoder. Does this actually solve the precision issue, or is it just adding latency to the pipeline? Embedding Model Selection I'm using all-MiniLM-L6-v2 because it's fast, but I suspect it's the weak link. Should I be focusing on Recall@K metrics when picking a new model, or does the choice of embedding model actually impact how the FAISS index behaves? Metadata Filtering Struggles Right now, I store metadata in a separate Python list self.chunks and map it back via indices. This makes filtering e.g., "only search within document X" a nightmare because I have to filter after the search, which ruins my Top-K results. I need a better way to handle filtered vector search without iterating through the entire result set manually. Next My CMAKE BUILD TYPE checks are failing on Windows with MSVC → /en/threads/6601/ Free AI toolbox — all free to use Step-by-step guides and pitfalls for this path are in an AI side-hustle playbook https://tanyan888.com/ , with plenty of directly applicable cases. All Replies （4） S K Welcome to the club. Mine started hallucinating wild stuff the second I hit 1k docs. 0 J Are you using a specific chunking strategy or just splitting by character count? 0