{"slug": "we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-don-t", "title": "We put an AI helper in our course and spent weeks teaching it to say I don't know", "summary": "A developer spent weeks teaching an AI helper inside their course to say 'I don't know' after it confidently invented a nonexistent feature for a student. The project shifted from generating text to scoping knowledge, enforcing refusals, and handing off to humans, with a test suite to prevent regressions.", "body_md": "We wanted a helper inside our course that could answer student questions about the material. Getting it to answer was a weekend. You wire up a model, feed it the lesson content, students ask things, it responds. Demos great. Everyone claps.\n\nThen a student asked it something we had never covered, and it made up an answer. Confident, fluent, wrong. It invented a feature that does not exist and told the student to go use it. That was the moment the real project started, and the real project had almost nothing to do with generating text. It was about the opposite. Teaching the thing to stop.\n\nMost of the weeks after launch went into three problems. Scoping what it was allowed to know. Getting it to refuse instead of invent. And knowing when to get out of the way and hand a student to a person. I want to walk through each, because the honest version of building an AI helper is mostly this, not the demo.\n\nOur first instinct was to give the model everything. All the lessons, all the notes, every old thread. More context, better answers, right.\n\nWrong, mostly. When the knowledge was one big blob, the model treated a throwaway comment in an old draft with the same confidence as a core lesson. It could not tell canon from noise. So we cut it down. The helper only sees a defined set of source material, each chunk tagged with where it came from and how authoritative it is. A published lesson outranks a forum reply. A forum reply outranks nothing, because we stopped feeding it random forum replies.\n\nWe also made the boundary explicit in the system prompt. The helper is told, in plain terms, what subjects it covers and that everything else is out of scope. That sounds obvious, but before we wrote it down the model assumed its scope was \"anything a helpful assistant could discuss,\" which is the whole internet. Naming the fence mattered more than any retrieval trick we tried.\n\nThe other thing scoping bought us was traceability. Because every answer is built from tagged chunks, we can look at a bad answer and see exactly which source led it astray. Half of our fixes were not prompt changes at all. They were us finding one stale document and deleting it.\n\nGetting a model to refuse is weirdly hard, because the whole training gradient pushes it to be helpful. Ask it something outside its knowledge and its instinct is to help anyway, which means guess.\n\nWe attacked this from a few sides. In the prompt we gave it permission to not know, in strong language, with examples of good refusals. A refusal in our system is not a dead end. It says what it cannot answer, and it points the student somewhere real, usually a human. Giving the model a concrete \"I do not have that, here is what to do next\" template moved the needle more than telling it \"do not hallucinate,\" which models mostly ignore.\n\nThen we tested it like software, which is the part I would repeat on any project like this. We built a set of questions we know are out of scope, plus a set of near misses, questions that sound in scope but are not. Every change to the prompt or the sources runs against that set. If the refusal rate on the out of scope questions drops, the change does not ship. Before we had that test, every improvement to answering quietly made refusing worse, and we could not see the trade until a student did.\n\nOne detail that helped a lot. We stopped rewarding long answers. A model that feels it must fill space will invent space to fill. Once we told it that a short answer and a clean \"ask a mentor about this one\" were both fully acceptable outcomes, the inventing dropped. It had permission to be brief, so it stopped padding.\n\nThe last piece was accepting that the helper should lose some conversations on purpose. If a student is frustrated, or the question touches something personal like their progress or a refund, or the model has refused twice in a row, it hands off to a human. No third guess.\n\nWe wired the handoff as a real path, not a polite dead end. The helper passes along what was asked and what it could not cover, so the person picking it up is not starting cold. That context transfer is small engineering and large payoff. Students hate repeating themselves, and a handoff that drops everything they said feels worse than no helper at all.\n\nIf there is one thing I would tell someone starting this, it is that the hard, valuable work sits on the refusal side, not the answering side. Answering is a demo. Refusing well, scoping tightly, handing off cleanly, that is the product. It is slower and less fun to build, and it is the difference between a helper students trust and a confident liar sitting inside your course.\n\n*AGINE Academy is an independent product by AGINE AI (not affiliated with Anthropic). We teach building with Claude by doing the work, not watching lectures.*", "url": "https://wpnews.pro/news/we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-don-t", "canonical_source": "https://dev.to/academy_agineai/we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-dont-know-hfc", "published_at": "2026-08-22 01:00:00+00:00", "updated_at": "2026-08-22 01:14:50.599887+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-safety", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-don-t", "markdown": "https://wpnews.pro/news/we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-don-t.md", "text": "https://wpnews.pro/news/we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-don-t.txt", "jsonld": "https://wpnews.pro/news/we-put-an-ai-helper-in-our-course-and-spent-weeks-teaching-it-to-say-i-don-t.jsonld"}}