# How to build a RAG system from scratch in Python (chunk embed retrieve cite)( https://ai.studybydoing.in)

> Source: <https://dev.to/krish0549/how-to-build-a-rag-system-from-scratch-in-python-chunk-embed-retrieve-cite-fa4>
> Published: 2026-09-18 03:21:53+00:00

Most "RAG tutorials" hand you a framework and a `.from_documents()` one-liner, and you never actually see what happens inside. So I built one **by hand** — chunking, embeddings, a tiny

  vector store, hybrid retrieval, re-ranking, and cited generation — to understand each moving part. Here's the mental model and the two pieces that matter most.

## What RAG actually is

An LLM only knows what was in its training data. **RAG (Retrieval-Augmented Generation)** lets it answer questions about *your* private/current documents by retrieving relevant snippets at

  query time and putting them in the prompt. The model then answers from that supplied context — facts, not guesses.

The pipeline has two timelines:

In one line: **RAG = look things up first, then answer from what you found.** The offline row is a librarian shelving books; the online row is you asking a question and getting the right

  pages handed to you before you write your reply.

## The highest-leverage decision: chunking

Models retrieve **chunks, not whole documents** — so how you split matters more than almost anything else:

``` python
  import re

  def chunk_text(text, source, target_words=120, overlap=25):
      """Split on paragraphs, then pack into ~target_words chunks with overlap."""
      paras = [p.strip() for p in re.split(r"\n\s*\n", text) if p.strip()]
      chunks, buf = [], []
      for p in paras:
          buf.append(p)
          if sum(len(x.split()) for x in buf) >= target_words:
              chunks.append(" ".join(buf))
              buf = buf[-1:]  # carry last para as overlap
      if buf:
          chunks.append(" ".join(buf))
      return [{"text": c, "source": source, "id": f"{source}#{i}"} for i, c in enumerate(chunks)]
```

The rules that survived the labs:

## Embeddings + a vector store in ~15 lines

An embedding turns text into a vector where **similar meanings sit close together**. Store each chunk's vector; at query time, embed the question and find the nearest chunks by cosine

  similarity.

``` python
  import numpy as np
  from sentence_transformers import SentenceTransformer  # swap for any provider

  _model = SentenceTransformer("all-MiniLM-L6-v2")

  def embed(texts):
      # normalize -> cosine similarity collapses into a plain dot product
      return np.asarray(_model.encode(texts, normalize_embeddings=True))

  class VectorStore:
      def add(self, chunks):
          self.chunks = chunks
          self.vecs = embed([c["text"] for c in chunks])   # (N chunks x d)

      def search(self, query, k=4):
          q = embed([query])[0]
          sims = self.vecs @ q                             # similarity to every chunk, one step
          top = np.argsort(-sims)[:k]
          return [(self.chunks[i], float(sims[i])) for i in top]
```

Two things clicked for me here:

`embed()` is the one place text becomes numbers
## Then: hybrid retrieval, re-ranking, and *citations*

The last labs add the parts that separate a demo from something usable:

## The mental model to keep

If you want to *run* each stage yourself (the lesson has an in-browser Python terminal, no setup) the full build is here 👉 **[Build a RAG System From
  Scratch](https://ai.studybydoing.in/ch03-rag)**. It's part of a free course that builds RAG, agents, eval, and production LLM systems by hand:

**[ai.studybydoing.in](https://ai.studybydoing.in)**.