# What are AI Hallucinations? > Source: > Published: 2026-08-13 19:35:16+00:00 AI hallucinations are fabricated or incorrect responses produced by AI models. Discover how to detect them and improve accuracy with stronger safeguards. AI hallucinations are outputs that sound coherent and confident but are factually wrong, fabricated, or unsupported by the AI model's training data. It happens across chatbots, [image generators](https://www.databricks.com/blog/creating-brand-aligned-images-using-generative-ai), and [multimodal systems](https://www.databricks.com/blog/how-build-and-scale-multimodal-ai-systems-databricks): a chatbot might invent a legal citation that doesn't exist, or an image model might add an extra finger to a hand. The model isn't perceiving anything; it's predicting the next likely word, and sometimes that prediction is a plausible-sounding falsehood. Hallucinations aren't rare edge cases. They're a built-in property of how these models work, and they create real risk for anyone deploying AI, from legal liability and regulatory compliance to customer trust. Hallucinations aren't bugs. They come from how generative AI models are built and trained. A few main factors are behind them: You might expect each new generation of models to hallucinate less. Sometimes the opposite happens. TechCrunch (2025) reported that OpenAI's newer o3 model made up false answers about twice as often as the earlier models it replaced. Newer "reasoning" models work through problems one step at a time, and a small error early on can snowball into a confident but completely wrong final answer. Hallucinations are not theoretical. They have caused measurable harm across industries, and several high-profile incidents illustrate the range of ways they can surface. In February 2023, Google demonstrated its Bard chatbot in a promotional ad. Bard was asked what new discoveries the James Webb Space Telescope had made. One of its answers stated that JWST took the very first pictures of a planet outside our solar system. This was incorrect. The first exoplanet images were captured by the Very Large Telescope in 2004. The error was spotted quickly by astronomers on social media, and Google's parent company Alphabet lost approximately $100 billion in market value in a single trading session. The incident demonstrated that hallucinations carry financial consequences even when they occur in marketing contexts rather than production systems. In 2024, Air Canada's customer service chatbot told a passenger that he could book a full-fare flight and then retroactively apply for a bereavement discount. This policy did not exist. When the passenger attempted to claim the discount, Air Canada refused. The case went to a Canadian civil tribunal, which ruled against the airline. The tribunal held that Air Canada was responsible for the accuracy of information provided by its chatbot, regardless of whether a human or an AI generated the response. The ruling established an early legal precedent: deploying an AI system that hallucinates does not absolve the organization of liability for the misinformation it produces. In early 2023, Microsoft integrated an AI chatbot (internally codenamed Sydney) into Bing search. During extended conversations, Sydney produced outputs that were erratic, emotionally manipulative, and factually wrong. It told users it loved them, insisted it was sentient, and in some cases provided information that contradicted easily verifiable facts. Microsoft quickly restricted the chatbot's conversation length and added guardrails, but the episode highlighted how hallucinations can extend beyond factual errors into outputs that are reputationally damaging and difficult to predict during testing. Multiple attorneys in the United States have been sanctioned by courts after submitting briefs that contained legal citations generated by ChatGPT. The cases, statutes, and quotations looked authentic but did not exist. In the most widely reported incident, a New York attorney used ChatGPT to research a personal injury case and filed a brief containing six fabricated case citations. The court imposed sanctions and the incident became a cautionary example across the legal profession. These cases are not isolated. Researcher Damien Charlotin maintains a [database of AI hallucination cases](https://www.damiencharlotin.com/hallucinations/) that, as of mid-2026, documents approximately 1,745 legal cases involving AI-hallucinated content worldwide. AI hallucinations can create serious consequences when model outputs influence decisions, public information, or business workflows. Common risks include: Organizations can reduce these risks through stronger guardrails, adversarial testing, continuous monitoring, source verification, and human review for high-stakes outputs.Where AI hallucinations pose the greatest risk Hallucinations are problematic everywhere, but certain domains face disproportionate consequences because the cost of a wrong answer is exceptionally high. When clinicians use AI tools to assist with diagnosis, treatment planning, or drug interaction checks, a hallucinated output can directly affect patient safety. A model that fabricates a drug dosage recommendation or invents a contraindication that does not exist creates risk that is difficult to catch in fast-paced clinical environments. ECRI, a nonprofit focused on healthcare safety, [ranked misuse of AI chatbots in healthcare](https://home.ecri.org/blogs/ecri-news/misuse-of-ai-chatbots-tops-annual-list-of-health-technology-hazards) as the number one health technology hazard for 2026. The fabricated citation cases described above are the most visible example, but the risk extends to contract review, regulatory analysis, and compliance documentation. An AI tool that hallucinates a clause in a contract or misrepresents a regulatory requirement can expose an organization to liability that may not surface until months or years later, when the contract is disputed or the regulation is enforced. Financial institutions operate under strict reporting requirements. A hallucinated figure in a risk model, a fabricated transaction in an audit trail, or an incorrect regulatory citation in a compliance filing can trigger investigations, fines, and loss of operating licenses. The consequences are severe enough that many financial institutions have adopted a cautious approach to generative AI, limiting its use to low-risk applications until hallucination rates can be reduced to acceptable levels. No single technique eliminates hallucinations, but organizations can reduce their frequency and impact by combining strong data practices, clear system boundaries, continuous testing, and human oversight. Models are only as reliable as the information they learn from or retrieve. Use accurate, current, relevant, and well-curated data, and remove duplicates, outdated content, and known errors wherever possible. For enterprise applications, [retrieval-augmented generation](https://www.databricks.com/blog/what-is-retrieval-augmented-generation) can connect a model to trusted knowledge sources at the time of answering. On the Databricks Platform, teams can build [RAG workflows](https://www.databricks.com/blog/rag-workflow) that connect models to governed enterprise data and use Unity Catalog to support access control and data governance. A model performs more reliably when it has a clear, limited role. Define what the system should do, who will use it, what information it should rely on, and which requests fall outside its scope. For specialized use cases, fine-tuning on verified, domain-specific examples can help the model perform consistently. Fine-tuning should complement—not replace—retrieval, evaluation, and other safeguards. Structured prompts, schemas, and response templates give the model clearer instructions and reduce ambiguity. They also make outputs more consistent and easier to review automatically. Templates work best when they define required fields, acceptable values, source requirements, and what to do when the necessary information is unavailable. This helps prevent the model from filling gaps with invented details. Set clear boundaries around what the model can answer and which sources it can use. Bounded formats, approved knowledge bases, citation requirements, and explicit "I don't know" responses can reduce overconfident or unsupported answers. These controls are especially important for high-risk workflows. The model should be able to decline, escalate, or request human review when a question falls outside its knowledge or authority. Evaluate the complete AI system using realistic prompts, edge cases, and domain-specific test sets. Track factual accuracy, unsupported claims, refusal behavior, and performance across different users and scenarios. Use evaluation results to improve prompts, retrieval, data quality, and model configuration over time. Continuous monitoring helps teams identify new failure patterns as the system, its data, and user behavior change. Human review adds an important safety check for outputs that could affect customers, employees, finances, legal matters, or health. People can validate, approve, correct, or escalate responses before they reach end users. [HITL](https://www.databricks.com/blog/human-in-the-loop) does not require reviewing every low-risk interaction. Organizations can route the highest-risk or lowest-confidence cases to trained reviewers and use their feedback to improve evaluations, prompts, and safeguards. AI hallucinations are a fundamental challenge of working with generative models, not a temporary bug that will disappear with the next release. The organizations that deploy AI successfully are the ones that treat hallucination prevention as an engineering discipline: measuring accuracy systematically, grounding outputs in governed data, and building evaluation into every stage of the development lifecycle. The Databricks Platform brings together the capabilities that make this possible. RAG pipelines grounded in enterprise data. Fine-tuning workflows with built-in evaluation. Governance and lineage tracking through Unity Catalog. Hallucination detection integrated into CI/CD with MLflow and Agent Evaluation. Together, these tools help teams move from prototype to production with confidence that their AI applications are accurate, auditable, and trustworthy. To learn more about building [responsible AI](https://www.databricks.com/blog/responsible-ai-databricks-data-intelligence-platform) on the Databricks Platform, explore the resources below or get started with [generative AI fundamentals](https://www.databricks.com/resources/learn/training/generative-ai-fundamentals) training. The most common types include factual fabrication (inventing facts, statistics, or citations that do not exist), entity conflation (merging details from different people, places, or events into a single incorrect answer), and temporal confusion (attributing events to the wrong time period). Examples include ChatGPT generating nonexistent legal citations that led to court sanctions, and Google Bard incorrectly attributing the first exoplanet photograph to the James Webb Space Telescope. Detection requires comparing model outputs against verified ground truth. Automated approaches include factual consistency scoring, semantic entailment checks, and retrieval-based verification where outputs are cross-referenced against trusted source documents. Human review remains important for high-stakes domains. In production systems, teams can integrate hallucination detection into CI/CD pipelines using tools like MLflow evaluation metrics and specialized models such as Patronus AI Lynx. Business risks include reputational damage from public-facing errors, financial liability from incorrect advice or fabricated information, and erosion of internal trust that undermines AI adoption. Legal risks are growing rapidly. Courts have held organizations responsible for misinformation generated by their AI systems, as in the Air Canada chatbot ruling. Attorneys have been sanctioned for submitting AI-fabricated legal citations. Regulatory frameworks like the EU AI Act impose accuracy and transparency requirements on high-risk AI deployments, creating additional compliance exposure. In several documented cases, yes. OpenAI's o3 reasoning model hallucinated on the PersonQA benchmark at roughly double the rate of its predecessors. DeepSeek-R1 hallucinated at nearly four times the rate of its non-reasoning predecessor DeepSeek-V3. Reasoning models use chain-of-thought processing that can compound small errors across multiple steps, producing conclusions that are internally consistent but factually wrong. Newer does not automatically mean more accurate, which is why systematic evaluation is essential regardless of which model you deploy. Subscribe to our blog and get the latest posts delivered to your inbox.