How to build a RAG system from scratch in Python (chunk embed retrieve cite)( https://ai.studybydoing.in)

Most "RAG tutorials" hand you a framework and a .from_documents() one-liner, and you never actually see what happens inside. So I built one by hand — chunking, embeddings, a tiny

vector store, hybrid retrieval, re-ranking, and cited generation — to understand each moving part. Here's the mental model and the two pieces that matter most.

What RAG actually is #

An LLM only knows what was in its training data. RAG (Retrieval-Augmented Generation) lets it answer questions about your private/current documents by retrieving relevant snippets at

query time and putting them in the prompt. The model then answers from that supplied context — facts, not guesses.

The pipeline has two timelines:

In one line: RAG = look things up first, then answer from what you found. The offline row is a librarian shelving books; the online row is you asking a question and getting the right

pages handed to you before you write your reply.

The highest-leverage decision: chunking #

Models retrieve chunks, not whole documents — so how you split matters more than almost anything else:

  import re

  def chunk_text(text, source, target_words=120, overlap=25):
      """Split on paragraphs, then pack into ~target_words chunks with overlap."""
      paras = [p.strip() for p in re.split(r"\n\s*\n", text) if p.strip()]
      chunks, buf = [], []
      for p in paras:
          buf.append(p)
          if sum(len(x.split()) for x in buf) >= target_words:
              chunks.append(" ".join(buf))
              buf = buf[-1:]  # carry last para as overlap
      if buf:
          chunks.append(" ".join(buf))
      return [{"text": c, "source": source, "id": f"{source}#{i}"} for i, c in enumerate(chunks)]

The rules that survived the labs:

Embeddings + a vector store in ~15 lines #

An embedding turns text into a vector where similar meanings sit close together. Store each chunk's vector; at query time, embed the question and find the nearest chunks by cosine

similarity.

  import numpy as np
  from sentence_transformers import SentenceTransformer  # swap for any provider

  _model = SentenceTransformer("all-MiniLM-L6-v2")

  def embed(texts):
      return np.asarray(_model.encode(texts, normalize_embeddings=True))

  class VectorStore:
      def add(self, chunks):
          self.chunks = chunks
          self.vecs = embed([c["text"] for c in chunks])   # (N chunks x d)

      def search(self, query, k=4):
          q = embed([query])[0]
          sims = self.vecs @ q                             # similarity to every chunk, one step
          top = np.argsort(-sims)[:k]
          return [(self.chunks[i], float(sims[i])) for i in top]

Two things clicked for me here:

embed() is the one place text becomes numbers

Then: hybrid retrieval, re-ranking, and citations #

The last labs add the parts that separate a demo from something usable:

The mental model to keep #

If you want to run each stage yourself (the lesson has an in-browser Python terminal, no setup) the full build is here 👉 Build a RAG System From Scratch. It's part of a free course that builds RAG, agents, eval, and production LLM systems by hand:

ai.studybydoing.in.