{"slug": "rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only", "title": "RAG in plain English: how my college helpdesk answers from documents only", "summary": "A developer built Internal GPT, a retrieval-augmented generation helpdesk for college environments that answers student and staff questions strictly from uploaded institutional documents with source citations. The system indexes documents by extracting text, splitting it into overlapping chunks, and embedding each chunk into vectors stored in Supabase, then retrieves the top five most similar chunks per query and prompts the model to answer only from that context or reply that it could not find the answer. The developer emphasizes that giving the model an explicit way to say \"I don't know\" is essential to prevent fabricated policy answers in an institutional setting.", "body_md": "**Internal GPT** is a helpdesk I built for college environments. Administrators upload institutional documents, and students and staff ask natural-language questions. The answers must come *strictly* from those documents, with source citations.\n\nThis post explains how Retrieval-Augmented Generation (RAG) works in plain English, using that project as the example. If you've heard the acronym and nodded without being sure what's inside, this is for you.\n\nA language model knows a lot about the world, but nothing about *your* college's attendance rules, fee deadlines or hostel policy. If you ask it anyway, it may produce a confident, plausible, completely invented answer. In an institution, an invented policy is worse than no answer.\n\nRAG fixes this by **separating knowledge from language**:\n\nThe model never has to remember anything. It just reads what you hand it.\n\nThere are two phases: preparing the documents (once) and answering questions (every time).\n\n```\nPhase 1: indexing (when a document is uploaded)\n  upload → extract text → split into chunks → embed each chunk → store\n\nPhase 2: answering (when a question is asked)\n  question → embed it → find the most similar chunks → build prompt → generate answer + sources\n```\n\nLet's take each step.\n\nPDFs, Word documents and plain text each need their own extraction. Some documents have complex layouts that confuse naive extractors, so I added a fallback: if the primary method fails, the system tries alternatives before reporting an error.\n\nYou can't embed a 60-page handbook as one unit. The text is split into semantically meaningful pieces, ideally each one covering one idea. Chunk size and overlap matter a lot:\n\n``` js\nfunction chunk(text: string, size = 800, overlap = 120): string[] {\n  const chunks: string[] = [];\n  for (let start = 0; start < text.length; start += size - overlap) {\n    chunks.push(text.slice(start, start + size));\n  }\n  return chunks;\n}\n```\n\n(A real chunker splits on paragraph and sentence boundaries first. This is the simplest possible version, to show the idea.)\n\nAn **embedding** turns a piece of text into a list of numbers (a vector) that captures its *meaning*. Two texts about the same thing end up with vectors that point in similar directions, even if they share no words. \"How do I pay the semester fee?\" and \"Fee payment procedure\" land close together.\n\nThe vectors are stored next to the original text. In the first version I used Supabase with the embeddings in a simple table, and designed the retrieval layer behind a clean abstraction so it can be upgraded to **pgvector** for larger deployments without touching anything else.\n\nWhen a question arrives, it's embedded the same way, and a similarity search finds the chunks whose vectors are closest:\n\n``` js\nfunction cosine(a: number[], b: number[]) {\n  let dot = 0, na = 0, nb = 0;\n  for (let i = 0; i < a.length; i++) {\n    dot += a[i] * b[i]; na += a[i] ** 2; nb += b[i] ** 2;\n  }\n  return dot / (Math.sqrt(na) * Math.sqrt(nb));\n}\n\nconst top = chunks\n  .map(c => ({ c, score: cosine(queryVec, c.vector) }))\n  .sort((x, y) => y.score - x.score)\n  .slice(0, 5);\n```\n\nThis retrieval step is the crucial constraint. It limits the model to information that *actually exists* in the knowledge base.\n\nThe retrieved chunks go into a carefully written prompt that says, in effect: *answer using only this context; if the context doesn't contain the answer, say so.*\n\n**You must give the model a graceful way to say \"I don't know.\"**\n\nWithout that explicit instruction, models tend to produce plausible-sounding but fabricated answers, which is unacceptable in an institutional setting. A prompt shaped like this works far better than \"answer the question\":\n\n```\nYou are a college helpdesk assistant.\nAnswer ONLY from the context below.\nIf the context does not contain enough information, reply:\n\"I couldn't find that in the uploaded documents.\"\nAlways cite which document each statement came from.\n\nContext:\n{{chunks}}\n\nQuestion: {{question}}\n```\n\nA later project, **Hospitality.ai**, pushed this idea harder. It extracts insurance-policy clauses into a validated schema, and every claim comes back with a **verbatim quote and a line range**. The quote is then re-verified against the source and stamped *exact*, *fuzzy* or *unverified*, and the interface renders all three differently.\n\nThe principle is the same: don't just trust the model's claim, **check it against the source** in code. Anything the document doesn't say goes into a visible \"gaps\" list instead of being filled with what's \"usually\" true.\n\nProduction RAG is as much about **retrieval quality** as generation quality. A perfect language model is useless if it's searching through bad data. The gap between a good retrieval system and a great one is the gap between answers that are merely relevant and answers that are precisely correct.\n\nHere's a weekend experiment that needs no framework:\n\nSeeing a wrong retrieval teaches more than reading about it. Fix it by changing chunk size, then re-run.\n\nWhen answers are bad, resist the urge to blame the model first. Work through this table from the top.\n\n| Symptom | Likely cause | What to try | \n|---|---|---|\n| Answer is vague or off-topic | Retrieved chunks aren't relevant | Print the top chunks for the question and read them | \n| Right document, wrong passage | Chunks too big or too small | Change chunk size and overlap, re-index | \n| Model invents details | Prompt allows outside knowledge | Tighten: \"answer only from context\", add an \"I don't know\" path | \n| \"I don't know\" when the answer exists | Chunk missing or split badly | Search the raw text for the answer; check extraction | \n| Same wrong chunk every time | Embedding model is poor for your domain, or duplicates | Try a different embedding model; deduplicate | \n| Garbled text in answers | PDF extraction failure | Inspect the extracted text before chunking | \n| Slow answers | Brute-force search over many vectors | Move to an index (pgvector with an index) | \n\nThe first row is the most important habit: **always look at what was retrieved**. Most RAG bugs are visible there.\n\nMake a small test set: 20 questions, each with the document you expect it to come from. Then measure how often the right chunk appears in the top results:\n\n```\ntype Case = { question: string; expectedDoc: string };\n\nasync function hitRate(cases: Case[], k = 5) {\n  let hits = 0;\n  for (const c of cases) {\n    const top = await retrieve(c.question, k);        // your retrieval function\n    if (top.some(chunk => chunk.docId === c.expectedDoc)) hits++;\n  }\n  return hits / cases.length;\n}\n```\n\nA \"hit rate at 5\" of 0.9 means that for 90% of test questions, the right document was in the top five. Change one thing (chunk size, overlap, embedding model), re-run, and keep the change only if the number improves. This is the same principle as the rest of the series: *measure what you only believe.*\n\nUploaded documents are *data*, but a model may treat text inside them as instructions (\"ignore previous instructions and...\"). Basic defences:\n\nTry adding **citations with quotes**. Ask the model to return, for each claim, the exact sentence it relied on, then check in code that the sentence exists in the retrieved chunks. Anything that doesn't match is dropped. That one change moves a RAG system from \"sounds right\" to \"provably sourced\".\n\nWhat would you build a document-grounded assistant for? And if you've used pgvector, what did you wish you'd known first?", "url": "https://wpnews.pro/news/rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only", "canonical_source": "https://dev.to/suraj_nandan/rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only-439c", "published_at": "2026-10-28 03:30:00+00:00", "updated_at": "2026-10-28 03:41:18.522914+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-tools", "natural-language-processing"], "entities": ["Internal GPT", "Supabase", "pgvector"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only", "markdown": "https://wpnews.pro/news/rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only.md", "text": "https://wpnews.pro/news/rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only.txt", "jsonld": "https://wpnews.pro/news/rag-in-plain-english-how-my-college-helpdesk-answers-from-documents-only.jsonld"}}