{"slug": "an-ai-legal-assistant-that-refuses-to-cite-law-it-can-t-verify", "title": "An AI legal assistant that refuses to cite law it can't verify", "summary": "A team of developers built AILegal, a legal assistant that enforces a rule of never stating a legal fact it hasn't verified against an official source. The system stores each of its 53 norms as a structured record with a source URL, source hash and human-set verified flag, and a scheduled job re-fetches every official source every three days to detect changes. The team reported that a soft paraphrase classifier reached roughly 57% precision and is therefore restricted to asking clarifying questions rather than producing verdicts.", "body_md": "We built AILegal — a legal assistant for people who can't afford a lawyer — around a single rule: **never state a legal fact we haven't verified against the official source.** This post is about the engineering that makes the rule enforceable, and about what still doesn't work.\n\nEarly versions used the usual approach: a careful prompt, a corpus in the context window, \"answer only from these sources\". It failed in ways that are easy to miss if you don't check:\n\nNeither answer *looked* wrong. That's the dangerous part: a plausible citation is worse than an obvious refusal.\n\nEvery norm in the corpus is a record, not a paragraph in a prompt:\n\n```\nNorm(\n    id=\"us-cares-4024c\",\n    code=\"15 U.S.C. § 9058(c)\",\n    jurisdiction=\"us\",\n    summary=\"...\",              # plain-language, human-reviewed\n    source_url=\"https://...\",   # official text\n    source_hash=\"...\",          # hash of the checked version\n    checked_at=\"2026-10-18\",\n    verified=True,              # human review passed\n)\n```\n\nThe corpus currently has **53 norms**: 11 Russian, 26 US, 9 UK, 7 Canadian.\n\nA scheduled job runs **every three days**. It fetches every official source, requires that the numbers and terms from our summary actually appear in the fetched text, and stores a hash of the version it checked. If the hash changes, the norm is flagged for a human re-check.\n\n`verified` is deliberately *not* set by the machine. The script proves \"the text says what we claim\"; a human decides \"this norm is usable for a deadline\". Those are different claims, and conflating them is how legal AI ends up inventing law.\n\nAt answer time the rule is boring on purpose: if a norm isn't verified, the assistant does not name deadlines or article numbers derived from it. It says what it can't confirm and asks for the missing fact.\n\nWe also measured a \"soft\" classifier that catches paraphrases by word-stem similarity. The exact stage resolved 12 of 31 test paraphrases; the soft stage added hints for 14 more, of which 8 were correct — roughly 57% precision. That is nowhere near enough to answer a legal question: every other hint would be about a different problem. So the soft stage only asks a clarifying question. It never produces a verdict.\n\nSynonyms. \"вычисли\" vs \"посчитай\", \"списали\" vs \"удержали\" — different words, same meaning. Without a source of synonymy, similarity scoring can't tell them apart. We tried local embedding models (`nomic-embed-text`, `qwen3-embedding`) on a canonical paraphrase; the results were top-3/top-4 with cosine gaps of 0.001–0.03 — noise. The real fix is not a better model but a semantic representation of the norms themselves: everyday search phrasings, written and reviewed per norm. That's legal work, not a prompt tweak.\n\nIf you ask a legal assistant \"what's the deadline?\" and it answers confidently, you can't tell a verified answer from a hallucinated one. The gate makes the failure mode boring: \"I can't confirm this from the official text.\" Boring is what you want when the cost of a wrong answer is a missed court deadline.\n\nTry to break it: [ailegal.legal](https://ailegal.legal) — take any norm, compare it with the source it links to, and try to make the assistant state something the source doesn't.\n\n*Not legal advice. Legal content is RU/EN; the interface speaks more languages.*", "url": "https://wpnews.pro/news/an-ai-legal-assistant-that-refuses-to-cite-law-it-can-t-verify", "canonical_source": "https://dev.to/mrpkk/an-ai-legal-assistant-that-refuses-to-cite-law-it-cant-verify-2cje", "published_at": "2026-10-19 14:41:58+00:00", "updated_at": "2026-10-19 14:44:29.636222+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "ai-safety", "large-language-models", "natural-language-processing"], "entities": ["AILegal", "nomic-embed-text", "qwen3-embedding"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/an-ai-legal-assistant-that-refuses-to-cite-law-it-can-t-verify", "markdown": "https://wpnews.pro/news/an-ai-legal-assistant-that-refuses-to-cite-law-it-can-t-verify.md", "text": "https://wpnews.pro/news/an-ai-legal-assistant-that-refuses-to-cite-law-it-can-t-verify.txt", "jsonld": "https://wpnews.pro/news/an-ai-legal-assistant-that-refuses-to-cite-law-it-can-t-verify.jsonld"}}