Your AI chatbot isn't hallucinating. It's reading your outdated docs.

A developer built Knowledge Freshness Audit, a free, MIT-licensed Claude skill that scans attached SOPs, policies, FAQs and help articles for contradictions, undated files, duplicates, missing owners and unanswered questions, returning a prioritised fix list rather than deciding which fact is correct. Tested on four small documents with planted problems, the skill flagged all of them, and the test documents and full report are included in the repo's example/ folder. The tool deliberately avoids copying personal data into its report, referring to files and lines instead.

When a support chatbot gives a wrong answer, the first instinct is to blame the model. But when I looked at how these failures are described in support and operations communities, a different pattern kept showing up: the model was faithfully repeating the documents it was given. The documents were the problem. Typical examples: A chatbot can't resolve a contradiction. It picks whichever page it finds first, so the answer becomes a coin flip. New employees fail in exactly the same way. I built a Claude skill called Knowledge Freshness Audit . You attach your SOPs, policies, FAQs, or help articles, and it returns a prioritised fix list: It never decides which fact is correct. If one page says 7 days and another says 14, the skill flags it and asks. Only the business knows its current rule; an AI guessing here would just create a third version of the truth. It keeps personal data out of the report. It refers to files and lines instead of copying customer names, phone numbers, or account details. I tested it on four small documents with planted problems: two contradictions refund window and support email , three undated files, one duplicate, three files with no owner, one "TBD" answer, and one unanswered question. The skill found all of them. The test documents and the full report are in the repo's example/ folder, so you can check the output yourself. It's free and MIT-licensed: https://github.com/sopkits/knowledge-freshness-audit https://github.com/sopkits/knowledge-freshness-audit Install the .skill file in Claude requires a plan that supports Skills , attach your docs, and say: "Audit these help docs - our chatbot keeps giving wrong answers." I'd love feedback - especially on problems it misses. This article was written with AI assistance and reviewed by me before publishing.