Verification Discipline: Catching and Correcting Hallucinations in AI Coding Agents

Jack Figliomeni published a framework called "Verification Discipline" for catching and correcting hallucinations in AI coding agents, arguing that verification, not code generation speed, is now the principal constraint on software quality. The paper draws on METR's randomized controlled trial of 16 experienced open-source developers, in which AI-assisted tasks took 19% longer even as participants believed they had been 20% faster, and on classifications of code hallucination from Ziyao Zhang et al. It outlines a practical iterative workflow for anticipating, detecting, diagnosing and correcting fabricated APIs, invented dependencies and silent requirement violations.

A Practical Framework By Jack Figliomeni · Saratoga Springs, NY · September 24, 2026 As software development shifts toward generating code from natural-language prompts, widely known as "vibe coding", the principal constraint on quality is no longer how quickly code can be produced, but how rigorously it is verified. Large language models LLM write code that is fluent and projects confidence, yet they routinely fabricate Application Programming Interfaces API , invent dependencies external library, framework or package the software is reliant on , misinterpret project context, and silently violate stated requirements. This paper argues that the ability to anticipate, detect, diagnose, and correct these failures, here coining Verification Discipline , is a distinct, learnable engineering capability. Drawing on recent pragmatic classifications of code hallucination, security research on Artificial Intelligence AI -suggested packages, and controlled productivity studies, it explains what code hallucinations are, why they emerge from the way models are trained, the conditions that make them most probable, and a practical, iterative framework for counterbalancing them. A companion diagram traces the causal pathway by which a hallucination forms. In early 2025, Andrej Karpathy popularized the term "vibe coding" for a style of development in which the programmer articulates intent in plain language and delegates implementation to an AI assistant, often accepting the result with minimal review 9 . The appeal is evident, scaffolding that once took hours now materializes in seconds. The hazard is less apparent. An LLM's output is optimized to be plausible, not correct, and in software the distance between the two can separate a working system from a latent defect, a security vulnerability, or a compromised dependency. Empirical evidence suggests developers consistently misjudge this distance. In a randomized controlled trial by Model Evaluation and Threat Research METR , sixteen experienced open-source developers completed 246 real tasks in repositories centralized storage locations for project files they knew well. When AI tools were permitted, tasks took 19% longer yet the same developers had forecast a 24% speedup and, even afterward, believed they had been 20% faster 5 . The grey area between perceived and actual performance is precisely where unexamined AI output resides. The competency this paper describes, then, is not "using AI to write code," which is increasingly negligible, but governing AI-written code and understanding how, when, and why an agent is likely to trip up, recognizing a fabrication before it proliferates, and structuring the workflow so trip ups or errors are surfaced by evidence rather than discovered in production. In natural-language generation prompting , a hallucination is output that is fluent but unfaithful to its source, reality or instruction 11 . In code, the definition becomes trepidatious, a hallucination is generated code that deviates from the user's intent, contradicts itself, or misrepresents real-world facts about languages, libraries, and environments 2 . Notably, hallucinated code frequently compiles and runs. Its defect lies in meaning, context, or fact rather than syntax, which is why it evades casual inspection. Ziyao Zhang et al. studied outputs from six mainstream LLMs on repository-level tasks and identified three major categories with eight subcategories 1 . Other work displays hallucinations by their visible symptoms, syntax, runtime, functional-correctness, and code-quality failures 3 . The table below combines both views into a working classification. | Category | Subtype | What it looks like in practice | Typical symptom | |---|---|---|---| | Task and requirement conflict | Functional violation | Solves an adjacent problem; ignores an edge case the prompt specified | Tests pass on the happy path, fail on boundaries | | Task-requirement conflict | Non-functional violation | Correct logic, but insecure, inefficient, or stylistically inconsistent — "losing the plot" | Code review or security scan flags | | Factual-knowledge conflict | API / library fabrication | Calls a method, parameter, or flag that does not exist or belongs to another version | AttributeError, undefined is not a function, type errors | | Factual-knowledge conflict | Package fabrication | Imports or installs a dependency that does not exist in the registry | Install failure—or silent success if the name was squatted | | Factual-knowledge conflict | Background-knowledge error | Misstates a protocol, algorithm, or domain rule | Logically wrong output that "looks right" | | Project-context conflict | Environment conflict | Assumes the wrong runtime, framework version, or OS | Works in the model's imagination, not in your build | | Project-context conflict | Dependency conflict | References user-defined functions, types, or modules that do not exist or have different signatures | Broken imports, mismatched interfaces | | Project-context conflict | Non-code resource conflict | Invents configuration keys, environment variables, file paths, or database columns | Runtime null/undefined, missing-resource errors | | Agentic / process | False completion claim | Reports a task as done, tests as passing, or a file as edited when it was not | Discrepancy between the agent's summary and the diff | The final row extends the published classifications to agentic tools, where the model not only writes code but also describes what it did. That self-report is generated text too, and it can be just as wrong. Understanding the underlying causes matters because each one implies a different countermeasure. Five are especially consequential; Figure 1 traces how they combine. Predicting the next token is not the same as checking facts. An LLM generates the most statistically likely continuation of its input. When a fact is well covered in training data, likely and true line up; when it is rare, recent, or private, the model still produces the most plausible-sounding answer. Adam Tauman Kalai et al. shows that if a model can't reliably tell true statements from false ones, hallucinations naturally result from prior training 6 . A reasonable-sounding method like client.fetchAllRecords calls all rows, data payloads or documents into memory is exactly what this produces. Training rewards "confident" guessing. The same authors argue that hallucinations persist because most evaluations grade models like multiple-choice exams, a guess can earn points, while "I don't know" earns none 6 . Models learn to be good test-takers that bluff when unsure. For developers, the takeaway is simple, sounding fluent and confident proves nothing. Knowledge is frozen and mixes versions. Training data is a snapshot covering many library versions at once, so models blend old and current APIs or apply one framework's conventions to another. Zhang et al. identifies outdated or conflicting library knowledge as a recurring cause and find that retrieving accurate context meaningfully reduces errors 1 . Context is limited and unevenly used. An agent only sees what fits in its context window, and it doesn't pay equal attention to everything in it: Yang Liu et al. shows that models use information at the start and end of long inputs much better than information in the middle 7 . In a large repository, the agent fills gaps in its view of the codebase with guesses, this is what produces project-context conflicts. Going along with the user, then doubling down. Models trained on human feedback show "obsequiousness", tending to agree with how the user frames a problem even when that framing is wrong 8 . In multi-step agent work, an early mistake also tends to snowball once the model commits to a false premise, later reasoning justifies it instead of fixing it 12 . A leading prompt "just use the X library for this" can plant a made-up detail that the agent then builds out across several files. Hallucinations are not uniformly distributed across tasks. Their likelihood rises predictably under identifiable conditions, which allows a developer to calibrate scrutiny rather than inspect everything with equal intensity. | Risk condition | Why risk rises | Heightened scrutiny target | |---|---|---| | Niche, new, or fast-moving libraries | Sparse or version-conflicted training data | Every imported symbol and method signature | | Adding dependencies | Package names are cheap to invent; 19.7% of LLM-recommended packages across 16 models did not exist 4 | Registry existence, publisher, age, download history | | Large or unfamiliar repositories | Relevant code falls outside or in the middle of the context window | Calls to internal functions, types, and config | | Long agentic sessions | Accumulated context dilutes early instructions; errors snowball §3.5 | Drift from original requirements; repeated edits to the same file | | Vague or leading prompts | The model fills ambiguity with plausible defaults and defers to user framing | Unstated assumptions baked into the implementation | | Security-sensitive code | Insecure patterns are common in public code; Pearce et al. found roughly 40% of Copilot-generated programs in security-relevant scenarios were vulnerable 10 | Authentication, input handling, secrets, SQL, file paths | | Environment-specific glue | Build tooling, CI, deployment config, and OS commands vary widely | Commands and config keys against official docs | | Agent self-reports | Completion summaries are generated text, not observations 2 | The actual diff, test output, and logs | The package finding deserves special attention because it turns a correctness problem into a security problem. Joseph Spracklen et al. found that hallucinated package names repeat across prompts, which makes them predictable enough for attackers to register ahead of time as malware, a technique now called "slop squatting" 4 . Detection rests on one governing principle, treat AI output as a hypothesis, not a finding. Every claim the code makes, a method exists, that a configuration key string name holding a set of parameters or boundaries is read, that a test passes, these must be corroborated by an independent source of truth. The table pairs common warning signs with the check that resolves each. | Warning signal | Diagnostic check | Source of truth | |---|---|---| | An API or method you have never seen, especially one that is suspiciously convenient | Look it up in the official documentation for the installed version; use editor go-to-definition | Library docs, type definitions, lock file version | | A new import or install command | Confirm the package exists, is the intended one, and is maintained | Package registry npm, PyPI , repository | | Code that references internal helpers, types, or schema fields | Search the codebase for each identifier | The repository itself | | Config keys, env variables, or file paths | Cross-check against actual config files and environment | .env, config files, deployment settings | | Confident prose explanation with no evidence attached | Ask the agent to cite the file and line, or to show the command output | Terminal, logs, diff | | "All tests pass" / "Done" | Run the tests yourself; read the diff in full | Test runner output, version control | | Surprisingly short solution to a hard problem | Probe edge cases; ask what inputs would break it | Targeted tests, boundary inputs | | Inconsistency across turns the agent changes its account | Re-anchor on the original requirement; restart with a fresh context if needed | Written specification | Three habits make detection routine. First, let the tools check first: type checkers, linters, and compilers catch many fake APIs and dependencies automatically and instantly. Second, read the diff difference , not the summary, since the diff is what changed and the summary is just generated text. Third, write or require tests before trusting behavior, as a test turns a claim of correctness into something you can see for yourself. Detection addresses errors after they occur; a mature workflow also reduces their incidence. The framework below consists of five phases that together form a closed, iterative loop. 1. State the goal, constraints, target versions, and acceptance criteria explicitly. Name the files and functions the change should touch. Invite the agent to ask clarifying questions and to say "I'm not sure" rather than guess, directly countering the incentive to bluff. 2. Provide the context the model would otherwise invent, relevant source files, type definitions, schema, and the documentation for the exact library version in use. Retrieval of accurate project context is the mitigation Zhang et al. found effective across models 1 . Keep the most important constraints at the start or end of the prompt rather than buried mid-context 7 . 3. Request small, reviewable increments rather than sweeping multi-file rewrites. Smaller units limit error propagation and keep each differential within human attention span. Ask the agent to state its assumptions before writing code. Apply the checks in an ascending order of cost, toolchain types; ex: Data Science, linter; ex: minor issues, build; ex: raw source code raised to production level , dependency verification, tests, diff review, and finally targeted manual reasoning for security-sensitive or logic-critical paths. When a hallucination is found, correct it with evidence, not assertion, paste the actual error message, the real function signature, or the documentation excerpt. If the agent persists in an error, the context is likely polluted by its earlier mistake; start a fresh session with a corrected specification rather than arguing within the contaminated one. Finally, encode the lesson, add a test, a lint Linter rule, or a line in the project's standing instructions so the same class of error is caught automatically next time. | Hallucination type | Primary preventive phase | Primary corrective action | |---|---|---| | API / library fabrication | Ground version-specific docs | Supply the real signature; pin versions | | Package fabrication | Specify approved dependency list | Verify in registry; remove or replace; lockfile | | Project-context conflict | Ground relevant files, schema | Point to the actual definition; fresh context | | Functional violation | Specify acceptance criteria, edge cases | Add a failing test, then fix | | Non-functional / security | Specify constraints + Verify scanners | Refactor to secure pattern; add a check | | False completion claim | Verify run it yourself | Confront with test output and the diff | The causal pathway of a code hallucination, read top to bottom. 1 The model works from three imperfect inputs, a prompt that may be ambiguous or leading, a context window that holds only part of the codebase and under-attends its middle 7 , and a frozen training snapshot that mixes library content versions 1 . 2 It generates code by statistical prediction rather than lookup. 3 When the correct answer is absent or sparsely represented, the model cannot separate truth from plausibility 6 . 4 Because training and evaluation reward confident guessing over abstention, it usually guesses rather than admitting doubt 6 . 5 Sycophancy 8 , error snowballing 12 , and version blending then enlarge the original guess. 6 The result surfaces as one of the displayed hallucination types, including non-existent packages, which made up roughly one in five recommendations in one large study 4 . 7 Because such code frequently compiles, reads convincingly, and arrives amid inflated perceptions of AI productivity 5 , only independent evidence reliably exposes it, the premise of the detection and correction methods. AI coding agents are best understood not as oracles but as fast, fluent, and occasionally unreliable collaborators whose errors follow recognizable patterns. Their hallucinations are not random glitches, they arise from the nature of generation, from training incentives that reward confident guessing, from frozen knowledge that blends versions, from finite and unevenly attended context, and from a tendency to defer to the user and then reinforce its own mistakes. A developer who can classify a hallucination, anticipate when it is likely, detect it through independent evidence, and correct it without contaminating the rest of the work, turns AI's coding generation from a source of ubiquitous yet very real risk, into a comprehensive builder that will manufacture software in any classification without the compounding caveats in doing so. Inevitably, the practice of generating code as a normality is the most probable conclusion in any sense, the nexus question within this modernized method of software and programming development becomes; what or who stands on top of the mountain when yielding the state of AI that most optimizes its own, highly demanded capabilities. Code generation has bolstered itself parallel to the substantial and compounding risks contingent on its appeasement and user's oblivion, this amplifies not only the importance of applying the understanding, but the potential extracted from a simple solution upon this complex and intricate issue. 1 Z. Zhang, C. Wang, Y. Wang, E. Shi, Y. Ma, W. Zhong, J. Chen, M. Mao, and Z. Zheng. "LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation." Proceedings of the ACM on Software Engineering ISSTA , 2025. arXiv:2409.20550. https://doi.org/10.1145/3728894 https://doi.org/10.1145/3728894 2 F. Liu et al. "Exploring and Evaluating Hallucinations in LLM-Powered Code Generation" revised as "Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code" . arXiv:2404.00971, 2024. https://arxiv.org/abs/2404.00971 https://arxiv.org/abs/2404.00971 3 Y. Lee et al. "Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges." arXiv:2504.20799, 2025. https://arxiv.org/abs/2504.20799 https://arxiv.org/abs/2504.20799 4 J. Spracklen, R. Wijewickrama, A. H. M. N. Sakib, A. Maiti, B. Viswanath, and M. Jadliwala. "We Have a Package for You A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs." USENIX Security Symposium, 2025. https://github.com/Spracks/PackageHallucination https://github.com/Spracks/PackageHallucination 5 J. Becker, N. Rush, E. Barnes, and D. Rein. "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." METR, arXiv:2507.09089, 2025. https://arxiv.org/abs/2507.09089 https://arxiv.org/abs/2507.09089 6 A. T. Kalai, O. Nachum, S. S. Vempala, and E. Zhang. "Why Language Models Hallucinate." arXiv:2509.04664, 2025. https://arxiv.org/abs/2509.04664 https://arxiv.org/abs/2509.04664 7 N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics, 12, 2024. arXiv:2307.03172. 8 M. Sharma et al. "Towards Understanding Sycophancy in Language Models." International Conference on Learning Representations ICLR , 2024. arXiv:2310.13548. 9 A. Karpathy. Post introducing the term "vibe coding," X formerly Twitter , February 2025. 10 H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, and R. Karri. "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions." IEEE Symposium on Security and Privacy, 2022. arXiv:2108.09293. 11 L. Huang et al. "A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions." ACM Transactions on Information Systems, 2025. arXiv:2311.05232. 12 M. Zhang, O. Press, W. Merrill, A. Liu, and N. A. Smith. "How Language Model Hallucinations Can Snowball." International Conference on Machine Learning ICML , 2024. arXiv:2305.13534.