# Stop trusting LLM 'hallucinated success'

> Source: <https://dev.to/renato_marinho/stop-trusting-llm-hallucinated-success-cmj>
> Published: 2026-08-18 01:59:55+00:00

I've seen this loop dozens of times. An AI agent tells you it finished the refactor. It claims the tests passed. It swears the new environment variable is correctly set in the `.env`

file.

You look at the terminal, or better yet, you check the repo yourself, and everything is a mess.

The fundamental issue isn't that LLMs aren't smart; it's that they suffer from a lack of objective closure. They operate on probabilistic next-token prediction. When an agent finishes a sequence of reasoning, its internal probability distribution peaks at "Success". But in engineering, probability doesn't mean anything—only state does.

If you want to build reliable autonomous workflows, you cannot rely on an agent saying "Done" as proof of work. You need a feedback loop rooted in deterministic truth.

When we talk about Model Context Protocol (MCP), most people focus on how to get data *into* the model (fetching docs, querying databases). We spend so much time building eyes for the AI that we forget to build hands that can actually touch and verify what they just did.

Most current implementations fall into the trap of trusting the agent's self-assessment. This leads to exactly what I call 'phantom completions': tasks that appear done according to the conversation history but failed in the filesystem or violated a runtime constraint.

A proper verification layer needs to act as a judge, not just another participant in the chat. It needs to move away from linguistic confirmation toward empirical validation.

To solve this, I’ve been looking at how we bridge task descriptions with verifiable reality. It’s not about asking the AI "Did you do it?", it’s about providing tools that allow it (or a supervisor process) to prove it via three distinct vectors: presence, content integrity, and environmental health.

There is a specific implementation designed precisely for this purpose: [the Claude Task Completion Verifier](https://vinkius.com/ai-agent-connect/claude-task-completion-verifier). Unlike generalist assistants, this MCP server focuses entirely on closing the loop using math and system state rather than vibes.

Here is how it fundamentally changes the workflow:

**1. File Presence and Pattern Matching ( check_file_integrity)**

`package.json`

, you force a regex match against that specific key within that specific file. If the pattern doesn't hit, the task isn't complete. Period.**2. Environmental Truth ( execute_environment_check)**

**3. Full Audit Loops ( verify_task_completion)**

You might ask: "Why can't I just tell Claude to check its own work?"

You technically can, but you won't fix the bias inherent in many models toward affirmative responses once they believe they have followed instructions successfully. To break out of that cycle, you need tool definitions that are decoupled from conversational intent.\lyet having tools that provide 'mathematical completion percentages' moves even closer to engineering rigor than pure text analysis ever could.

The [Claude Task Completion Verifier](https://vinkius.com/ai-agent-connect/claude-task-completion-verifier) works differently because it treats task fulfillment as an assertion test rather than a dialogue component.

instead of trying to improve the LLM's sense of duty through prompting alone,

you augment its capability with deterministic gates.

catalogue link: [https://vinkius.com/ai-agent-connect/claude-task-completion-verifier](https://vinkius.com/ai-agent-connect/claude-task-completion-verifier)

however after implementing these kinds of verifiers)

it becomes significantly easier to scale multi-agent orchestrations without needing human intervention at every single step to confirm basic sanity checks.\r$\

*MCPs are the music of AI Agents. We built the catalog. Discover Vinkius MCP Catalog.*