Large Language Models (LLMs) like GPT-4, Claude, Llama 3, and Gemini are astonishing. They can write essays, generate code, translate languages, and hold coherent conversations. But if you have tried to use them in a real business context, you have probably hit the same three walls that everyone else does:
Every LLM is trained on data up to a certain date. Ask GPT-4 about a product launched last week, and it either says “I don’t know” or — worse — invents an answer. This is fine for chit-chat, catastrophic for enterprise use.
LLMs are next-token predictors, not truth engines. When they don’t know something, they don’t stop — they guess fluently. A hallucinated legal citation, medical dose, or financial figure isn’t just embarrassing; it’s a liability.
Your internal wiki, customer support tickets, PDFs of contracts, Confluence pages, Jira comments — none of that is in the LLM. And you probably don’t want to fine-tune a model every time a document changes.
RAG (Retrieval Augmented Generation) is the elegant, practical, and now-industry-standard answer to all three problems.
Retrieval Augmented Generation (RAG) is a technique that combines two components:
-
A Retriever — that fetches relevant information from an external knowledge source (documents, databases, APIs) at query time.
-
A Generator — an LLM that uses the retrieved information as context to produce a grounded, accurate answer.
Imagine a very smart student taking an open-book exam.Without RAG → the student answers from memory alone (may forget, may guess).**With RAG **→ the student first flips to the exact right page of the textbook, reads it, then answers.
The student (LLM) is the same. The difference is having the right page open in front of them.
RAG = Retrieve relevant context + Feed it to an LLM + Generate a grounded answer.
That’s it. Everything else in this article is a variation, optimization, or evaluation of that flow.
Let’s be concrete about what RAG fixes:
RAG doesn’t make the LLM smarter. It makes the LLM better informed — and in enterprise settings, that’s usually what you actually need.
Update a document → the next query uses the new version. No retraining.
Answers are anchored to retrieved sources, dramatically reducing hallucinations.
Every answer can be shown with its citations. Auditors love this. So do users.
Fine-tuning a 70B model is expensive. Adding a PDF to a vector DB is free.
Your data never leaves your infrastructure (if you self-host the vector DB and the model). GDPR / HIPAA-friendly.
Swap the retriever, swap the LLM, swap the embedding model — each piece is independent.
You can literally show the retrieved chunks that produced the answer.
RAG isn’t a lab curiosity. It’s already in production at thousands of companies. Here are the patterns you’ll see everywhere:
An internal chatbot that answers: “What’s our parental leave policy?” by searching Confluence, Google Drive, and Notion.
Ingest all past tickets, product manuals, and FAQs. Deflect 40–70% of L1 tickets with grounded, cited answers.
Query thousands of contracts: “Find all agreements where the termination clause requires more than 90 days notice.”
Ground answers in latest medical guidelines and journals — never in the LLM’s training data.
Analyze earnings reports, SEC filings, and news to answer: “What did the CFO say about margins last quarter?”
Code-aware chat over your monorepo: “How do we implement retries in this codebase?”
Beyond keyword: “A waterproof jacket good for hiking in cold rain under $200.”
Personalized tutors grounded in the specific curriculum, not the internet.
These three are often confused. Let’s untangle them.
What it is: Find documents whose meaning matches the query (using embeddings) — not just keyword match.Output: A ranked list of documents.Analogy: A very smart Google.
What it is: Semantic Search + an LLM that reads the results and writes an answer.Output: A natural language answer with (optionally) citations.Analogy: A smart Google that also reads the top links and summarizes them for you.
What it is: Update the LLM’s weights on your domain data so the knowledge is baked in.Output: A new, specialized model.Analogy: Sending your intern to a 6-month bootcamp.
Rule of thumb: Fine-tuning teaches the model new skills. RAG gives the model new knowledge.
A production RAG system has two distinct phases:
Let’s walk through each stage with the “why” behind the “what”.
Get the raw text out of PDFs, Word docs, HTML, Markdown, databases, APIs, etc.
LLMs have context limits; vector search works best on focused passages. Split the text into ~200–1000 token chunks.
Convert each chunk into a high-dimensional vector using an embedding model. Similar meanings → nearby vectors.
Save the vectors (plus the original text and metadata) in a vector database that supports fast similarity search.
At query time, embed the user’s question and find the top-k most similar chunks.
Stuff those chunks into a prompt: “Answer the question using ONLY this context: {chunks}”.
Send the prompt to the LLM. Return the answer (plus citations).
Every step has knobs. Tuning those knobs is the art of RAG.
If your document is 100 pages long and you embed it as a single vector, that vector represents an average meaning — useless for precise retrieval. Chunking creates many focused vectors, each representing one idea.
Too small (< 100 tokens): Loses context. “The company reported a loss.” Which company?Too large (> 1500 tokens): Dilutes relevance. LLM may miss the key sentence.Sweet spot: ~300–800 tokens with 50–100 token overlap.
from langchain.text_splitter import RecursiveCharacterTextSplittertext = """Retrieval Augmented Generation (RAG) combines the strengths of retrieval-basedand generation-based approaches. It first retrieves relevant documents from aknowledge base and then uses a language model to generate an answer.RAG helps reduce hallucinations because the model is grounded in real sources.It also allows for easy updates: change the documents, and the answers update."""splitter = RecursiveCharacterTextSplitter( chunk_size=200, chunk_overlap=40, separators=["\n\n", "\n", ". ", " ", ""],)chunks = splitter.split_text(text)for i, c in enumerate(chunks): print(f"--- Chunk {i} ---\n{c}\n")
An embedding is a list of numbers (e.g., 1536 floats) that represents the meaning of a piece of text.
"dog" → [0.12, -0.44, 0.98, …]"puppy" → [0.14, -0.41, 0.95, …] (very close to "dog")"submarine" → [-0.71, 0.03, 0.22, …] (far from "dog")
The magic: semantically similar texts → geometrically close vectors.
python
Popular open embedding models:
`BAAI/bge-large-en-v1.5` - top-tier English`intfloat/e5-large-v2` - strong general-purpose`sentence-transformers/all-MiniLM-L6-v2` - tiny, fast, decent`nomic-ai/nomic-embed-text-v1.5` - long context (8k)
A vector DB stores vectors and supports Approximate Nearest Neighbor (ANN) search: given a query vector, find the top-k most similar stored vectors — fast, at billion-scale.
Cosine similarity — most common; measures angle between vectors.Dot product — fast; good when vectors are normalized.Euclidean (L2) distance — geometric distance.
Retrieval is the most impactful stage of RAG. Garbage retrieval → garbage generation, no matter how good your LLM is.
Given a query $q$ and a corpus of chunks C = {c_1, c_2, …, c_n}, return the top-k chunks that maximize: relevance(q, c_i)
Relevance is usually measured by embedding similarity, but can also include keyword overlap, metadata filters, recency, etc.
Dense retrieval: Uses embeddings. Great for semantic meaning (“car” ≈ “automobile”).Sparse retrieval: Uses keyword statistics (BM25, TF-IDF). Great for exact matches, product codes, names.Hybrid retrieval: Combines both. Almost always beats either alone.
Vector search doesn’t have to be blind. You can pre-filter by metadata:
collection.query( query_texts=["quarterly revenue"], n_results=5, where={"year": 2025, "doc_type": "earnings_call"},)
This is huge for multi-tenant apps, security scoping, and time-based filtering.
Typical k = 3–10.
Vector search at scale doesn’t do exact nearest neighbor — that’s O(n) per query and would take seconds for a million vectors. Instead, we use Approximate Nearest Neighbor (ANN) algorithms.
The dominant algorithm today. Builds a multi-layer graph:
Top layers are sparse (long-range links).
Bottom layer contains all points.
Search descends from the top, greedily hopping toward the query.
Result: logarithmic search time with >95% recall.
Cluster vectors into buckets. At query time, search only the nearest few buckets.
Compress vectors into small codes to save memory. Often combined with IVF (IVF-PQ) for billion-scale search.
You rarely tune ANN parameters directly, but knowing they exist helps you understand:
Why recall isn’t 100%.
Why ef_search or nprobe parameters exist.
Why raising them improves quality but slows things down.
The default. Embed query → find top-k nearest chunks.
results = collection.query(query_texts=["What is RAG?"], n_results=3)
Balances relevance with diversity. Prevents returning 5 nearly identical chunks.
from langchain_community.vectorstores import Chromafrom langchain_openai import OpenAIEmbeddingsvs = Chroma(persist_directory="./cdb", embedding_function=OpenAIEmbeddings())docs = vs.max_marginal_relevance_search( "How does RAG reduce hallucinations?", k=4, # final results fetch_k=20, # candidates to consider lambda_mult=0.5, # 0 = max diversity, 1 = max relevance)
python
Weaviate, Qdrant, and Elasticsearch support hybrid search natively.
Generate several rephrasings of the user’s query, retrieve for each, then merge.
from langchain.retrievers.multi_query import MultiQueryRetrieverfrom langchain_openai import ChatOpenAIllm = ChatOpenAI(model="gpt-4o-mini", temperature=0)retriever = MultiQueryRetriever.from_llm( retriever=vs.as_retriever(), llm=llm,)docs = retriever.invoke("How does RAG reduce hallucinations?")
Embed small chunks for precise matching, but return their larger parent chunks for richer context.
-
Ask the LLM to hallucinate a fake answer to the query.
-
Embed the fake answer.
-
Use that embedding to retrieve.
Surprisingly effective — the fake answer often looks more like the target chunk than the raw query.
A minimal RAG prompt:
You are a helpful assistant. Answer the user's question using ONLY theprovided context. If the answer is not in the context, say "I don't know."Context:{retrieved_chunks}Question: {user_question}Answer:
Number the chunks so the LLM can cite them: [1] … [2] …Explicitly forbid hallucination: “If not in context, say you don’t know.”** Ask for citations**: “Cite the chunk numbers you used.”** Cap total context tokens** to avoid overflow.Include metadata (source, date) inside each chunk header.
from openai import OpenAIclient = OpenAI()def build_prompt(question: str, chunks: list[str]) -> str: ctx = "\n\n".join(f"[{i+1}] {c}" for i, c in enumerate(chunks)) return f"""You are a precise assistant. Use ONLY the context below.If the answer is not present, reply "I don't know".Cite sources like [1], [2].Context:{ctx}Question: {question}Answer:"""def rag_answer(question: str, chunks: list[str]) -> str: prompt = build_prompt(question, chunks) resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": prompt}], temperature=0.1, ) return resp.choices[0].message.content
Now let’s put it all together into a working end-to-end system.
pip install langchain langchain-openai langchain-community \chromadb pypdf sentence-transformers tiktoken
python
import osos.environ["OPENAI_API_KEY"] = "sk-…"
python
from langchain_community.document_s import PyPDFfrom langchain.text_splitter import RecursiveCharacterTextSplitterfrom langchain_openai import OpenAIEmbeddings, ChatOpenAIfrom langchain_community.vectorstores import Chromafrom langchain.chains import RetrievalQA# 1. LOAD = PyPDF("company_handbook.pdf")pages = .load()# 2. CHUNKsplitter = RecursiveCharacterTextSplitter(chunk_size=600, chunk_overlap=100)chunks = splitter.split_documents(pages)# 3. EMBED + STOREembeddings = OpenAIEmbeddings(model="text-embedding-3-small")vectorstore = Chroma.from_documents( chunks, embeddings, persist_directory="./chroma_db",)# 4. RETRIEVE + GENERATEllm = ChatOpenAI(model="gpt-4o-mini", temperature=0)qa = RetrievalQA.from_chain_type( llm=llm, retriever=vectorstore.as_retriever(search_kwargs={"k": 4}), return_source_documents=True,)result = qa.invoke({"query": "What is our remote work policy?"})print("Answer:", result["result"])print("\nSources:")for doc in result["source_documents"]: print("-", doc.metadata.get("source"), "page", doc.metadata.get("page"))
That’s a working RAG system in ~25 lines. Everything else is optimization.
Both are Python frameworks for building LLM apps. Both do RAG well. They differ in philosophy.
Strengths: Massive ecosystem, agents, tools, integrations with everything.Weakness: Can feel over-abstracted; API churn.Best for: Complex chains, agent workflows, mixed tool use.
Strengths: Laser-focused on RAG. Best-in-class indexing primitives, query engines, evaluation.Weakness: Less flexible for non-RAG tasks.Best for: Data-heavy RAG apps, structured document QA.
Notice how much less ceremony there is — that’s LlamaIndex’s design goal.
Building a RAG-first product? LlamaIndex.
Building a multi-tool agent where RAG is one capability? LangChain.
Doing both? Use them together — they compose fine.
Real-world data is messy. Here’s how to handle it.
from langchain_community.document_s import PyPDF, UnstructuredPDF# Simple text extractiondocs = PyPDF("file.pdf").load()# Better: preserves tables, structuredocs = UnstructuredPDF("file.pdf", mode="elements").load()
python
from langchain_community.document_s import WebBasedocs = WebBase(["https://example.com/blog/post"]).load()
python
from langchain_community.document_s import CSVdocs = CSV("data.csv").load()
Each has a dedicated in langchain_community.document_s. Real production systems usually build incremental sync on top: track last-modified timestamps, re-embed only changed docs.
from langchain_community.document_s import Directory, Textdocs = Directory("./docs", glob="**/*.md", _cls=Text).load()
Two patterns:
-
Text-to-SQL (agent generates SQL from natural language).
-
Row-to-Document (embed each row’s textual columns).
Always attach metadata during :
for doc in docs: doc.metadata.update({ "source": doc.metadata.get("source", "unknown"), "team": "engineering", "ingested_at": "2026-07-28", "access_level": "internal", })
Metadata enables filtering, security, and citations.
Let’s build a slightly more serious index.
from langchain_openai import OpenAIEmbeddingsfrom langchain_community.vectorstores import Chromafrom langchain.text_splitter import RecursiveCharacterTextSplitterfrom langchain_community.document_s import Directory, PyPDF = Directory( "./corpus", glob="**/*.pdf", _cls=PyPDF, show_progress=True,)raw_docs = .load()splitter = RecursiveCharacterTextSplitter( chunk_size=800, chunk_overlap=120,)chunks = splitter.split_documents(raw_docs)for c in chunks: c.metadata["team"] = "product" c.metadata["version"] = "v2"emb = OpenAIEmbeddings(model="text-embedding-3-large")vs = Chroma.from_documents( chunks, emb, persist_directory="./prod_chroma")
retriever = vs.as_retriever( search_type="mmr", search_kwargs={ "k": 5, "fetch_k": 25, "lambda_mult": 0.6, "filter": {"team": "product"}, },)docs = retriever.invoke("What changed in v2?")
python
from langchain.retrievers import EnsembleRetrieverfrom langchain_community.retrievers import BM25Retrieverbm25 = BM25Retriever.from_documents(chunks)bm25.k = 5dense = vs.as_retriever(search_kwargs={"k": 5})hybrid = EnsembleRetriever( retrievers=[bm25, dense], weights=[0.4, 0.6],)docs = hybrid.invoke("SKU-2025-A pricing change")
Hybrid typically boosts recall on queries containing exact terms (IDs, names, codes).
Trade-offs to consider:Dimension (384 vs 1536 vs 3072) — bigger = more storage/compute.Context length — how many tokens per chunk it can handle.Domain — general vs. code-specialized vs. multilingual. Cost — API vs. self-host.
from sentence_transformers import SentenceTransformer, utilpairs = [ ("What is RAG?", "Retrieval Augmented Generation combines retrieval and LLMs."), ("What is RAG?", "The Eiffel Tower is in Paris."),]for name in ["all-MiniLM-L6-v2", "BAAI/bge-small-en-v1.5"]: m = SentenceTransformer(name) for q, d in pairs: e1, e2 = m.encode([q, d]) print(f"{name:35s} sim={util.cos_sim(e1, e2).item():.3f} '{d[:40]}...'")
Always use the same embedding model for indexing and querying. Mixing them silently destroys retrieval quality.
Let’s write a clean, production-ready augmented generation function.
from openai import OpenAIfrom langchain_community.vectorstores import Chromafrom langchain_openai import OpenAIEmbeddingsclient = OpenAI()vs = Chroma(persist_directory="./prod_chroma", embedding_function=OpenAIEmbeddings(model="text-embedding-3-large"))SYSTEM_PROMPT = """You are a precise, honest assistant.Rules:1. Answer using ONLY the provided context.2. If the answer isn't in the context, reply exactly: "I don't know based on the provided documents."3. Cite sources inline like [1], [2] using the chunk numbers.4. Be concise. Do not invent facts."""def format_context(docs) -> str: parts = [] for i, d in enumerate(docs, start=1): src = d.metadata.get("source", "unknown") page = d.metadata.get("page", "?") parts.append(f"[{i}] (source: {src}, page: {page})\n{d.page_content}") return "\n\n".join(parts)def rag(question: str, k: int = 5) -> dict: docs = vs.similarity_search(question, k=k) context = format_context(docs) resp = client.chat.completions.create( model="gpt-4o-mini", temperature=0.1, messages=[ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"}, ], ) return { "answer": resp.choices[0].message.content, "sources": [ {"source": d.metadata.get("source"), "page": d.metadata.get("page")} for d in docs ], }out = rag("What are the key benefits of RAG?")print(out["answer"])print("\nSources:", out["sources"])
This pattern — retrieve → format → prompt → generate → return with citations — is the beating heart of virtually every RAG app in production.
You can’t improve what you don’t measure. RAG evaluation has two axes:
-
Retrieval quality — did we fetch the right chunks?
-
Generation quality — did the LLM produce a good answer given those chunks?
Create ~50–200 golden examples:
eval_set = [ { "question": "What is our maternity leave policy?", "ground_truth": "26 weeks of paid leave for birthing parents.", "relevant_doc_ids": ["hr_policy.pdf#p12", "hr_policy.pdf#p13"], }, # ...]
Hit Rate @ k — was any relevant doc in the top-k?MRR (Mean Reciprocal Rank) — how high was the first relevant doc?Recall @ k — fraction of relevant docs retrieved.Precision @ k — fraction of retrieved docs that were relevant.NDCG — quality-weighted ranking.
def hit_rate(retrieved_ids, relevant_ids): return int(any(r in relevant_ids for r in retrieved_ids))def mrr(retrieved_ids, relevant_ids): for i, r in enumerate(retrieved_ids, start=1): if r in relevant_ids: return 1 / i return 0.0def recall_at_k(retrieved_ids, relevant_ids, k): top = set(retrieved_ids[:k]) return len(top & set(relevant_ids)) / max(len(relevant_ids), 1)
Generation is trickier — there’s no single “correct” answer. Common approaches:
BLEU / ROUGE / METEOR — n-gram overlap. Weak for open-ended answers.BERTScore — semantic similarity via embeddings. Better.
Use a strong LLM (GPT-4, Claude) to grade answers on:Faithfulness — is the answer supported by the context?Answer relevance — does it actually answer the question?Context relevance — was the retrieved context useful?Correctness — vs. a golden reference (if you have one).
def llm_judge(question, answer, context, model="gpt-4o"): prompt = f"""Grade this answer on 3 axes (1-5):- Faithfulness (only uses the context)- Relevance (answers the question)- ClarityQuestion: {question}Context: {context}Answer: {answer}Return JSON: {{"faithfulness": int, "relevance": int, "clarity": int, "reason": str}}""" resp = client.chat.completions.create( model=model, messages=[{"role": "user", "content": prompt}], response_format={"type": "json_object"}, temperature=0, ) return resp.choices[0].message.content
There’s no universally best retriever. Match the method to the query pattern.
methods = { "dense": vs.as_retriever(search_kwargs={"k": 5}), "mmr": vs.as_retriever(search_type="mmr", search_kwargs={"k": 5, "fetch_k": 20}), "hybrid": hybrid, # from earlier}for name, r in methods.items(): hits, mrrs = [], [] for row in eval_set: docs = r.invoke(row["question"]) ids = [d.metadata.get("chunk_id") for d in docs] hits.append(hit_rate(ids, row["relevant_doc_ids"])) mrrs.append(mrr(ids, row["relevant_doc_ids"])) print(f"{name:8s} HitRate={sum(hits)/len(hits):.2f} MRR={sum(mrrs)/len(mrrs):.2f}")
Iterate. Measure. Pick what wins on your data.
RAGAS is an open-source library that automates RAG evaluation using LLM-as-judge with well-defined metrics.
Faithfulness — is the answer grounded in the retrieved context?Answer Relevancy — does the answer address the question?Context Precision — how much of the retrieved context is relevant?Context Recall — did we retrieve everything needed?Answer Correctness — vs. a ground-truth answer.
-
Build a curated eval set of ~100 Q/A/context/ground-truth rows.
-
Run RAGAS after every change (new chunker, new embedder, new prompt).
-
Track scores over time — treat regressions like test failures.
Users are messy. Their queries are:
Too short → *pricing?*Too vague →
Better queries → better retrieval → better answers. It’s often the single highest-ROI improvement in a RAG system after basic hygiene.
from openai import OpenAIclient = OpenAI()REWRITE_PROMPT = """Rewrite the user's question to be a clear, standalone,search-friendly query. Fix spelling, expand acronyms, and add relevant terms.Return ONLY the rewritten query.Original: {q}Rewritten:"""def rewrite(query: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": REWRITE_PROMPT.format(q=query)}], temperature=0, ) return resp.choices[0].message.content.strip()print(rewrite("pricing?"))# → "What are the current pricing plans and subscription tiers?"
EXPAND_PROMPT = """Generate 3 different rephrasings of this question thatwould help retrieve relevant documents from a knowledge base.Return each on its own line.Question: {q}"""def expand(query: str) -> list[str]: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": EXPAND_PROMPT.format(q=query)}], temperature=0.3, ) return [line.strip("-•* \t") for line in resp.choices[0].message.content.splitlines() if line.strip()]queries = expand("How do I reset my password?")
Retrieve for each, merge, deduplicate.
DECOMPOSE_PROMPT = """Break this complex question into simpler sub-questions.Return one per line.Question: {q}"""def decompose(query: str) -> list[str]: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": DECOMPOSE_PROMPT.format(q=query)}], temperature=0, ) return [l.strip("-•* \t") for l in resp.choices[0].message.content.splitlines() if l.strip()]print(decompose("What's our refund policy and how does it compare to competitors?"))# → ["What is our refund policy?", "What are our competitors' refund policies?", ...]
HYDE_PROMPT = """Write a short, factual paragraph that would perfectly answerthis question, as if you had access to authoritative sources.Question: {q}Paragraph:"""def hyde_retrieve(query: str, k: int = 5): resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": HYDE_PROMPT.format(q=query)}], temperature=0.2, ) fake_answer = resp.choices[0].message.content return vs.similarity_search(fake_answer, k=k)
CONTEXTUAL_PROMPT = """Given the chat history, rewrite the follow-up questionto be a standalone question.Chat history:{history}Follow-up: {q}Standalone question:"""def contextualize(history: str, q: str) -> str: resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": CONTEXTUAL_PROMPT.format(history=history, q=q)}], temperature=0, ) return resp.choices[0].message.content.strip()
Even with good retrieval, the top-k list is often noisy. Re-ranking fixes this by using a more expensive but more accurate model to re-score the candidates.
Stage 1 (Retriever): Fast, bi-encoder embedding search. Recall-focused.Stage 2 (Re-Ranker): Slow, cross-encoder that scores (query, doc) pairs directly. Precision-focused.
A bi-encoder embeds query and doc independently, then compares vectors. A cross-encoder feeds both together into a transformer, which can reason about their interaction — much more accurate, but too slow to run on millions of docs. Two-stage is the best of both.
BAAI/bge-reranker-large — open source, strong.Cohere Rerank — hosted API, very good.ColBERT / ColBERTv2 — late-interaction, fast.LLM-as-reranker — use GPT-4 to score. Highest quality, highest cost.
RERANK_PROMPT = """Given a query and a passage, rate how relevant the passageis to answering the query on a scale from 0 to 10. Return only the number.Query: {q}Passage: {p}Score:"""def llm_score(query, passage): resp = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": RERANK_PROMPT.format(q=query, p=passage)}], temperature=0, ) try: return float(resp.choices[0].message.content.strip()) except ValueError: return 0.0def llm_rerank(query, docs, top_n=5): scored = [(d, llm_score(query, d.page_content)) for d in docs] scored.sort(key=lambda x: -x[1]) return [d for d, s in scored[:top_n]]
php
def advanced_rag(user_query: str, history: str = "") -> dict: # 1. Rewrite / contextualize query = contextualize(history, user_query) if history else rewrite(user_query) # 2. Multi-query expansion expanded = [query] + expand(query) # 3. Retrieve broadly candidates = [] seen = set() for q in expanded: for d in vs.similarity_search(q, k=15): key = d.page_content[:100] if key not in seen: candidates.append(d) seen.add(key) # 4. Re-rank top_docs = rerank(query, candidates, top_n=5) # 5. Generate context = format_context(top_docs) resp = client.chat.completions.create( model="gpt-4o-mini", temperature=0.1, messages=[ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {user_query}"}, ], ) return { "answer": resp.choices[0].message.content, "rewritten_query": query, "sources": [d.metadata for d in top_docs], }
This one function embodies most of what a serious RAG system does at query time.
RAG is not just a technique — it’s the bridge that turns generic LLMs into trustworthy, up-to-date, domain-specific assistants. Master it, and you master the single most valuable pattern in applied AI today.
Mastering Retrieval Augmented Generation (RAG): The Complete End-to-End Guide was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.