Delhi High Court Flags AI-Hallucinated Legal Rulings, Orders Strict Action

The Delhi High Court set aside an income tax appellate order after discovering it relied on six non-existent judicial precedents, a practice the court described as an increasing trend of 'hallucinated' citations. A bench of Justices Dinesh Mehta and Rajneesh Kumar Gupta warned that citing non-existent judgments, whether AI-generated or self-created, constitutes fraud on the justice system and called for strict disciplinary action against offenders.

July 29, 2026 , Inside AI — The Delhi High Court has set aside an income tax appellate order after discovering it relied on six judicial precedents that never existed, a practice the court described as an increasing trend of “hallucinated” citations. A bench of Justices Dinesh Mehta and Rajneesh Kumar Gupta warned that citing non-existent judgments, whether AI-generated or self-created, constitutes fraud on the justice system and called for strict disciplinary action against offenders. The ruling came in writ petitions filed by the Employees' Provident Fund Organisation EPFO challenging an order of the Commissioner of Income Tax Appeals . The EPFO argued that the appellate authority had based its decision on fabricated case law. When the matter was first heard in May, the court examined paragraph 6 of the impugned order and found six citations that could not be verified by legal researchers. The non-existent cases included: CIT v. Outstanding Industrial Enterprises, CIT v. Orissa Corporation P Ltd., CIT v. Jayachandran Textile Mills, CIT v. Veerabhadrappa and Sons, CIT v. Sree Meenakshi Mills Ltd., and CIT v. Oswal Agro Mills Ltd. The court directed the author of the order, a then Additional/Joint Commissioner Appeals -2 who has since retired, to explain the source. In an affidavit, the officer stated that due to heavy workload, he relied on an Inspector who supplied the citations, and he inserted them without verification. The bench clarified that adjudicating authorities are free to rely on their own reasoning and need not support every conclusion with precedent. However, an order founded on incorrect law cannot stand. The court set aside the appellate order and remanded the matter for a fresh decision. Because the author had retired, the court declined to order disciplinary proceedings but used the case to issue a broader warning. "We have experienced that the practice of citing non-existing or hallucinated citations is increasing day by day," the court observed. "The time has come, if an authority or court is found citing or relying upon non-existent judgments, whether generated through artificial intelligence or self-generated, a strict action including a disciplinary action be taken," the court warned. The judgment highlights a growing challenge in legal systems worldwide: the infiltration of AI-generated content into judicial processes. Large language models are known to "hallucinate" or fabricate plausible but false information, including legal citations. This phenomenon has been documented in multiple jurisdictions. A 2023 study in arXiv https://arxiv.org/abs/2305.10853 found that AI tools can produce convincing but entirely fictitious legal references, posing risks for self-represented litigants and overburdened professionals. The Delhi High Court's stance aligns with international responses. In 2023, a New York federal judge sanctioned lawyers for submitting a brief with fake ChatGPT-generated citations. The UK Judiciary https://www.judiciary.uk/guidance-and-resources/artificial-intelligence-ai-judicial-guidance/ has issued guidance on responsible AI use in courts, emphasizing verification. India’s legal system, with its massive caseload, is particularly vulnerable. The Supreme Court of India has been exploring AI for translation and legal research, but this incident underscores the need for robust safeguards. Human Oversight Fails as Legal AI Hallucinations Multiply The retired officer's admission points to systemic pressures. Heavy caseloads and reliance on support staff create gaps where unverified information can enter rulings. The court did not specify whether the Inspector used AI tools, but the term "hallucinated" directly invokes AI-generated falsehoods. Even without AI, the fabrication of citations is a serious ethical breach. The court's warning that such acts amount to fraud raises the stakes for all adjudicating bodies. The ruling also exposes a tension between efficiency and integrity. AI can expedite legal research, but without mandatory verification protocols, its outputs can corrupt decisions. The Delhi High Court’s decision to remand the case without disciplinary action, due to retirement, leaves open questions about accountability. However, the precedent now exists for future cases to trigger strict consequences. "As the author of the impugned order has since retired from service, we do not wish to pass any direction for initiation of disciplinary action against him," the court said. Legal technologists argue that the solution is not to ban AI but to implement verification layers. Some courts now require attorneys to disclose AI use and certify that citations are real. The Delhi High Court’s order may accelerate such reforms in India. Meanwhile, the EPFO case will be reheard, this time grounded in actual law.