Sometimes a model states something false with total confidence. No hedging, no “I’m not sure” — just a wrong answer delivered like a fact. That’s a hallucination.
It’s not lying, exactly. The model isn’t tracking truth at all. It’s producing text that looks like a correct answer would look.
Why hallucinations happen #
A model is trained to predict plausible next words, not to check facts against a database. When it doesn’t know something, it doesn’t stop and say so — it keeps generating the most likely-sounding continuation, true or not.
This shows up constantly in code: a method name that sounds right but was never added to the library, an import path that would make sense if the package existed, a config flag that matches the naming pattern of ten real flags but isn’t one of them.
Knowledge limits #
Every model has a training cutoff — a point after which it simply hasn’t seen anything. Ask about a library released after that date and it may guess based on similar, older libraries. The guess can be close enough to sound right and wrong enough to break your build.
Real coding examples #
A few patterns come up often enough to name:
- Importing a module that doesn’t exist, or importing a real module from the wrong package.
- Calling a method that exists on a similar class but not this one.
- Setting a config option that was valid in an older major version but got renamed or removed.
None of these throw an obvious red flag in the model’s output. They read exactly like correct code.
Verification mindset #
Treat what a model hands you as a draft, not a verdict. Run it. Let the compiler, the linter, or the test suite catch what the model couldn’t. If something depends on an external fact — a library’s current API, a tool’s exact flag names — check the docs instead of trusting recall.
The habit that actually pays off: run the code before you believe the code.
Other limitations #
A model will often agree with you even when your premise is wrong — ask a leading question and it may go along with the mistake baked into it. And oddly, some tasks that are trivial for a plain script — counting exact characters, doing precise arithmetic, following a rigid rule with no exceptions — can trip up a model that otherwise writes sophisticated code.
Previous: [AI Foundations 1 - How AI Models Work](/2026/09/13/how-ai-models-work/)
Next: [AI Foundations 3 - Tokens and Pricing](/2026/09/13/tokens-and-pricing/)