You've probably seen the term RAG everywhere lately — "RAG pipeline," "RAG chatbot," "build your own RAG app." It sounds complicated, but the idea behind it is actually pretty simple.
In this article, I'll explain RAG in plain language, then walk through how it works using a real project I built: Guidely, an internal knowledge assistant that answers questions using a company's own documents.
Large language models (like GPT or Claude) are trained on a huge amount of general knowledge, but they don't know about your specific data — your company's internal docs, your product manuals, your onboarding guides. They also can't be retrained every time a document changes; that's slow and expensive.
RAG solves this without retraining the model at all.
RAG means: before answering a question, first go find the relevant pieces of your own documents, and hand those to the AI along with the question.
That's it. "Retrieval" (finding the right information) + "Augmented Generation" (the AI answers using that information). Instead of the AI answering from memory alone, it answers using facts you hand it in the moment.
Let's break down the three things you need to make this work: chunking, embeddings, and vector search.
You can't hand an AI model an entire 200-page document and ask it to search through it efficiently. So the first step is splitting documents into smaller, manageable pieces called chunks.
In Guidely, I used a token-window chunker — it splits text based on a fixed number of tokens (roughly, pieces of words) per chunk, rather than just splitting by paragraph or sentence. This matters because:
A token-window approach gives you consistent, predictable chunk sizes, which makes the next steps more reliable.
Once you have chunks, you need a way to compare "how similar" two pieces of text are — for example, how similar is a chunk to the user's question?
Computers can't compare meaning directly, so each chunk gets converted into a list of numbers called an embedding (a vector). Text with similar meaning ends up with numbers that are mathematically close to each other, even if the wording is completely different.
For example, "How do I reset my password?" and "Steps to change your login credentials" would produce embeddings that are close together, because they mean roughly the same thing — even though the words barely overlap.
One thing I added in Guidely: a SHA-256 hash-based embedding cache. Generating embeddings costs time and money, so before creating a new embedding, I hash the chunk's content and check if it's already been embedded before. If it has, I reuse the cached version instead of generating it again. This alone cut down repeated work significantly, especially when documents get updated but most of the content stays the same.
Now that every chunk has a numeric embedding, you need a fast way to search through thousands (or millions) of them to find the ones closest to the user's question.
This is where a vector database comes in. In Guidely, I used FAISS (Facebook AI Similarity Search) — a library built specifically for searching through large sets of embeddings quickly.
The flow looks like this:
Here's the full RAG flow, using Guidely as the example:
1. Documents come in (company docs, guides, wikis)
↓
2. Chunker splits them into token-window chunks
↓
3. Each chunk is embedded (with caching to skip repeat work)
↓
4. Embeddings are stored in a FAISS index
↓
5. User asks a question
↓
6. Question is embedded and compared against the FAISS index
↓
7. Top matching chunks are retrieved
↓
8. Chunks + question are sent to the AI model
↓
9. AI generates an answer grounded in the actual documents
The backend for this in Guidely runs on FastAPI, with FAISS handling the vector search, and a React/Vite frontend for the chat interface.
RAG isn't magic — it's really just: break documents into chunks, turn them into searchable numbers, find the closest matches to a question, and let the AI answer using those matches. Once you see it broken down like this, it becomes a lot less intimidating to build yourself.
If you're building something similar or have questions about any part of the pipeline, drop a comment below.