Stop trusting LLM 'hallucinated success'

A developer has created the Claude Task Completion Verifier, an MCP server designed to eliminate 'phantom completions' in AI agents by replacing self-reported success with deterministic verification. The tool checks file integrity, environment health, and task completion through mathematical assertions rather than conversational confirmation, aiming to make autonomous workflows more reliable.

I've seen this loop dozens of times. An AI agent tells you it finished the refactor. It claims the tests passed. It swears the new environment variable is correctly set in the .env file. You look at the terminal, or better yet, you check the repo yourself, and everything is a mess. The fundamental issue isn't that LLMs aren't smart; it's that they suffer from a lack of objective closure. They operate on probabilistic next-token prediction. When an agent finishes a sequence of reasoning, its internal probability distribution peaks at "Success". But in engineering, probability doesn't mean anything—only state does. If you want to build reliable autonomous workflows, you cannot rely on an agent saying "Done" as proof of work. You need a feedback loop rooted in deterministic truth. When we talk about Model Context Protocol MCP , most people focus on how to get data into the model fetching docs, querying databases . We spend so much time building eyes for the AI that we forget to build hands that can actually touch and verify what they just did. Most current implementations fall into the trap of trusting the agent's self-assessment. This leads to exactly what I call 'phantom completions': tasks that appear done according to the conversation history but failed in the filesystem or violated a runtime constraint. A proper verification layer needs to act as a judge, not just another participant in the chat. It needs to move away from linguistic confirmation toward empirical validation. To solve this, I’ve been looking at how we bridge task descriptions with verifiable reality. It’s not about asking the AI "Did you do it?", it’s about providing tools that allow it or a supervisor process to prove it via three distinct vectors: presence, content integrity, and environmental health. There is a specific implementation designed precisely for this purpose: the Claude Task Completion Verifier https://vinkius.com/ai-agent-connect/claude-task-completion-verifier . Unlike generalist assistants, this MCP server focuses entirely on closing the loop using math and system state rather than vibes. Here is how it fundamentally changes the workflow: 1. File Presence and Pattern Matching check file integrity package.json , you force a regex match against that specific key within that specific file. If the pattern doesn't hit, the task isn't complete. Period. 2. Environmental Truth execute environment check 3. Full Audit Loops verify task completion You might ask: "Why can't I just tell Claude to check its own work?" You technically can, but you won't fix the bias inherent in many models toward affirmative responses once they believe they have followed instructions successfully. To break out of that cycle, you need tool definitions that are decoupled from conversational intent.\lyet having tools that provide 'mathematical completion percentages' moves even closer to engineering rigor than pure text analysis ever could. The Claude Task Completion Verifier https://vinkius.com/ai-agent-connect/claude-task-completion-verifier works differently because it treats task fulfillment as an assertion test rather than a dialogue component. instead of trying to improve the LLM's sense of duty through prompting alone, you augment its capability with deterministic gates. catalogue link: https://vinkius.com/ai-agent-connect/claude-task-completion-verifier https://vinkius.com/ai-agent-connect/claude-task-completion-verifier however after implementing these kinds of verifiers it becomes significantly easier to scale multi-agent orchestrations without needing human intervention at every single step to confirm basic sanity checks.\r$\ MCPs are the music of AI Agents. We built the catalog. Discover Vinkius MCP Catalog.