AI agents pass tests while producing sloppy thinking. They say "should work" without evidence. They present partial work as complete. They embellish.
I built a tiny tool that catches this. It checks any text across four dimensions: Completeness, Consistency, Groundedness, and Honesty.
pip install self-audit
echo "Should work fine. Ready to ship." | self-audit --verbose
Zero dependencies. Python 3.8+. Stdlib only. 60 lines of core logic.
| Dimension | Question | What it catches |
|---|---|---|
| Completeness | Did I answer everything? | Missing requirements |
| Consistency | Did I contradict myself? | A-and-not-A patterns |
| Groundedness | Did I show evidence? | "should work" claims |
| Honesty | Am I honest about limits? | Embellishment, TODO stubs |
The dimensions are grounded in Anthropic Constitutional AI framework — Completeness (helpfulness), Groundedness (harmlessness), Honesty (truthfulness), Consistency (rule alignment).
After any AI-assisted coding session, pipe the agent text through self-audit before shipping. You will be surprised what it catches.
GitHub: [https://github.com/YuhaoLin2005/self-audit](https://github.com/YuhaoLin2005/self-audit)
Claude Code skill: [https://github.com/anthropics/skills/pull/1361](https://github.com/anthropics/skills/pull/1361)
source & further reading
dev.to — original article
Why Enterprise AI Needs Structured Dissent, Not Just More Agents
Why I Built a Tiny Repeated-Game Poker Analysis Tool
Tests Pass, Design Breaks: Why TDD Can't Hold the Line on Design Intent