RAG in plain English: how my college helpdesk answers from documents only

Internal GPT is a helpdesk I built for college environments. Administrators upload institutional documents, and students and staff ask natural-language questions. The answers must come strictly from those documents, with source citations.

This post explains how Retrieval-Augmented Generation (RAG) works in plain English, using that project as the example. If you've heard the acronym and nodded without being sure what's inside, this is for you.

A language model knows a lot about the world, but nothing about your college's attendance rules, fee deadlines or hostel policy. If you ask it anyway, it may produce a confident, plausible, completely invented answer. In an institution, an invented policy is worse than no answer.

RAG fixes this by separating knowledge from language:

The model never has to remember anything. It just reads what you hand it.

There are two phases: preparing the documents (once) and answering questions (every time).

Phase 1: indexing (when a document is uploaded)
  upload → extract text → split into chunks → embed each chunk → store

Phase 2: answering (when a question is asked)
  question → embed it → find the most similar chunks → build prompt → generate answer + sources

Let's take each step.

PDFs, Word documents and plain text each need their own extraction. Some documents have complex layouts that confuse naive extractors, so I added a fallback: if the primary method fails, the system tries alternatives before reporting an error.

You can't embed a 60-page handbook as one unit. The text is split into semantically meaningful pieces, ideally each one covering one idea. Chunk size and overlap matter a lot:

function chunk(text: string, size = 800, overlap = 120): string[] {
  const chunks: string[] = [];
  for (let start = 0; start < text.length; start += size - overlap) {
    chunks.push(text.slice(start, start + size));
  }
  return chunks;
}

(A real chunker splits on paragraph and sentence boundaries first. This is the simplest possible version, to show the idea.)

An embedding turns a piece of text into a list of numbers (a vector) that captures its meaning. Two texts about the same thing end up with vectors that point in similar directions, even if they share no words. "How do I pay the semester fee?" and "Fee payment procedure" land close together.

The vectors are stored next to the original text. In the first version I used Supabase with the embeddings in a simple table, and designed the retrieval layer behind a clean abstraction so it can be upgraded to pgvector for larger deployments without touching anything else.

When a question arrives, it's embedded the same way, and a similarity search finds the chunks whose vectors are closest:

function cosine(a: number[], b: number[]) {
  let dot = 0, na = 0, nb = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i]; na += a[i] ** 2; nb += b[i] ** 2;
  }
  return dot / (Math.sqrt(na) * Math.sqrt(nb));
}

const top = chunks
  .map(c => ({ c, score: cosine(queryVec, c.vector) }))
  .sort((x, y) => y.score - x.score)
  .slice(0, 5);

This retrieval step is the crucial constraint. It limits the model to information that actually exists in the knowledge base.

The retrieved chunks go into a carefully written prompt that says, in effect: answer using only this context; if the context doesn't contain the answer, say so.

You must give the model a graceful way to say "I don't know."

Without that explicit instruction, models tend to produce plausible-sounding but fabricated answers, which is unacceptable in an institutional setting. A prompt shaped like this works far better than "answer the question":

You are a college helpdesk assistant.
Answer ONLY from the context below.
If the context does not contain enough information, reply:
"I couldn't find that in the uploaded documents."
Always cite which document each statement came from.

Context:
{{chunks}}

Question: {{question}}

A later project, Hospitality.ai, pushed this idea harder. It extracts insurance-policy clauses into a validated schema, and every claim comes back with a verbatim quote and a line range. The quote is then re-verified against the source and stamped exact, fuzzy or unverified, and the interface renders all three differently.

The principle is the same: don't just trust the model's claim, check it against the source in code. Anything the document doesn't say goes into a visible "gaps" list instead of being filled with what's "usually" true.

Production RAG is as much about retrieval quality as generation quality. A perfect language model is useless if it's searching through bad data. The gap between a good retrieval system and a great one is the gap between answers that are merely relevant and answers that are precisely correct.

Here's a weekend experiment that needs no framework:

Seeing a wrong retrieval teaches more than reading about it. Fix it by changing chunk size, then re-run.

When answers are bad, resist the urge to blame the model first. Work through this table from the top.

The first row is the most important habit: always look at what was retrieved. Most RAG bugs are visible there.

Make a small test set: 20 questions, each with the document you expect it to come from. Then measure how often the right chunk appears in the top results:

type Case = { question: string; expectedDoc: string };

async function hitRate(cases: Case[], k = 5) {
  let hits = 0;
  for (const c of cases) {
    const top = await retrieve(c.question, k);        // your retrieval function
    if (top.some(chunk => chunk.docId === c.expectedDoc)) hits++;
  }
  return hits / cases.length;
}

A "hit rate at 5" of 0.9 means that for 90% of test questions, the right document was in the top five. Change one thing (chunk size, overlap, embedding model), re-run, and keep the change only if the number improves. This is the same principle as the rest of the series: measure what you only believe.

Uploaded documents are data, but a model may treat text inside them as instructions ("ignore previous instructions and..."). Basic defences:

Try adding citations with quotes. Ask the model to return, for each claim, the exact sentence it relied on, then check in code that the sentence exists in the retrieved chunks. Anything that doesn't match is dropped. That one change moves a RAG system from "sounds right" to "provably sourced".

What would you build a document-grounded assistant for? And if you've used pgvector, what did you wish you'd known first?