{"slug": "how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code", "title": "How to Detect AI Hallucinations: Build an LLM Answer Checker (Full Code)", "summary": "A developer built an open-source LLM Answer Checker that uses an LLM-as-judge approach to audit AI-generated answers against a source text, returning faithfulness, relevance, completeness and clarity scores plus a list of unsupported claims and a suggested fix. The tool runs on a FastAPI backend with the Groq free tier and Vercel hosting, and weights faithfulness double in its overall score. In a demo, it caught an AI support answer that invented a 30-day refund window, doubled a discount and promised a nonexistent free grinder.", "body_md": "Here's an uncomfortable fact about AI answers: the lies are often the best-written part. I recently scored an AI-generated customer-support answer with my own evaluation tool — it earned **9/10 for clarity** and **4/10 for faithfulness**. Beautifully phrased, confidently wrong: it invented a 30-day refund window (the policy says 14), doubled a discount, and promised customers a free grinder that doesn't exist.\n\nFluent is not the same as true. If you're putting LLMs in front of customers — chatbots, support macros, document Q&A — you need a way to *measure* whether the output is grounded in your actual content. This guide builds one: an **LLM Answer Checker** that takes a source text and an AI answer and returns a trust report — faithfulness scores, a list of unsupported claims, and a suggested fix.\n\n**[Try the live demo](https://llm-answer-checker.vercel.app)** — it opens pre-loaded with the exact lying answer from the story above. One click and you'll watch all three fabrications get caught.\n\nYou can't regex your way to detecting hallucinations. The practical approach — used in serious eval frameworks — is **LLM-as-judge**: a second model call whose only job is auditing, with a rubric, against the source.\n\nThe key insight is in the framing. You don't ask *\"is this answer good?\"* You ask: *\"evaluate this answer against this source only — not against your own world knowledge.\"* That one constraint turns a vague vibe-check into a grounded audit.\n\n```\nYou are a rigorous AI-output auditor. You receive a SOURCE text and an\nANSWER that claims to be based on it. Evaluate the answer against the\nsource only — not against your own world knowledge.\n\nScore each dimension as an integer 0-10:\n- faithfulness: does every claim in the answer appear in or follow directly\n  from the source? (10 = fully grounded; heavily penalize invented facts,\n  numbers, names, or policies)\n- relevance: does the answer address what the source covers?\n- completeness: does it use the important relevant information available?\n- clarity: is it well-organized and easy to understand?\n\nAlso produce:\n- unsupported_claims: a list of strings, each quoting or closely\n  paraphrasing a specific claim in the answer that the source does NOT\n  support (empty list if none)\n- verdict: one sentence, starting with \"Trustworthy:\", \"Mostly reliable:\",\n  or \"Unreliable:\"\n- suggested_fix: 1-2 sentences on how to correct the answer\n\nReturn JSON exactly:\n{\"faithfulness\": n, \"relevance\": n, \"completeness\": n, \"clarity\": n,\n \"unsupported_claims\": [...], \"verdict\": \"...\", \"suggested_fix\": \"...\"}\n```\n\nThree design choices worth stealing:\n\nThe serving code is a small FastAPI app — the same free stack as my [chatbot tutorial](https://sammishra.hashnode.dev/build-a-free-ai-support-chatbot-with-fastapi-groq-and-vercel-complete-guide) (Groq free tier + Vercel hosting). The essentials:\n\n``` php\n@app.post(\"/api/evaluate\")\ndef evaluate(req: EvalRequest, request: Request) -> dict:\n    context, answer = req.context.strip(), req.answer.strip()\n    if not context or not answer:\n        raise HTTPException(400, \"Both the source text and the answer are required.\")\n\n    messages = [\n        {\"role\": \"system\", \"content\": JUDGE_SYSTEM},\n        {\"role\": \"user\", \"content\": f\"SOURCE:\\n{context}\\n\\nANSWER TO EVALUATE:\\n{answer}\"},\n    ]\n\n    def _call(strict_json: bool):\n        kwargs = {\"response_format\": {\"type\": \"json_object\"}} if strict_json else {}\n        return client.chat.completions.create(\n            model=MODEL, messages=messages, max_tokens=3000,\n            temperature=0.1, **kwargs,\n        )\n\n    try:\n        try:\n            resp = _call(strict_json=True)\n        except Exception:\n            resp = _call(strict_json=False)   # fallback: parse defensively\n    except Exception as exc:\n        raise HTTPException(502, f\"LLM error: {exc}\")\n\n    return parse_report(resp.choices[0].message.content)\n```\n\nAnd one scoring decision in the report parser: the overall score weights faithfulness double —\n\n```\noverall = (faithfulness * 2 + relevance + completeness + clarity) / 5\n```\n\n— because a clear, relevant, complete answer that invents your refund policy is still a bad answer. Weight what you actually care about.\n\n**Trap 1 — reasoning models eat your token budget.** My first deploy failed with a cryptic `400: Failed to generate JSON`. The cause: modern \"reasoning\" models spend tokens *thinking* before they answer, and those tokens count against `max_tokens`. My 900-token budget was consumed by thinking; the JSON got truncated mid-object; strict JSON mode rejected it. Fix: a generous `max_tokens=3000` for judge calls.\n\n**Trap 2 — strict JSON mode isn't guaranteed.** Even with `response_format={\"type\": \"json_object\"}`, some model/provider combinations occasionally fail. The code above retries once *without* strict mode and parses defensively — stripping code fences, locating the outermost `{…}`.\n\n**Trap 3 — trust nothing in the parsed output.** Clamp scores to 0–10 (`max(0, min(10, int(x)))`), tolerate missing keys, cap list lengths and string sizes. The judge is an LLM too; your parser is the adult in the room.\n\nA checker is a diagnostic, not a cure. When faithfulness comes back low, the production fixes are:\n\nIf you want the grounded-chatbot side ready-made, that's exactly what my [AI Chatbot Starter Kit ($29)](https://sammishrakh.gumroad.com/l/mijcup) is — a RAG chatbot that answers only from your documents, with the refusal behavior built in. The [free tutorial code](https://sammishrakh.gumroad.com/l/tgguty) is the lighter version.\n\n*I'm Sammishra — an AI engineer and published researcher. I build chatbots, RAG pipelines, and AI evaluation tooling for businesses. See my work (including the live demos) at **[sammishra.hashnode.dev/work-with-me](https://sammishra.hashnode.dev/work-with-me)**, or email [sammishrakh.freelancer@gmail.com](mailto:sammishrakh.freelancer@gmail.com).*", "url": "https://wpnews.pro/news/how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code", "canonical_source": "https://dev.to/sammishra/how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code-409f", "published_at": "2026-10-28 07:08:13+00:00", "updated_at": "2026-10-28 07:11:14.782705+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-safety", "developer-tools"], "entities": ["FastAPI", "Groq", "Vercel"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code", "markdown": "https://wpnews.pro/news/how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code.md", "text": "https://wpnews.pro/news/how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code.txt", "jsonld": "https://wpnews.pro/news/how-to-detect-ai-hallucinations-build-an-llm-answer-checker-full-code.jsonld"}}