Where do we draw the line on AI in court filings?

At least 2,000 legal cases have involved AI hallucinations, according to a review of court rulings that includes a 12 December 2025 decision in Couvrette (D. Or.) imposing $110,000 in fees and fines — the record total — for 15 fake cases and 8 fake quotations followed by concealment. The same review documents the Nebraska Supreme Court suspending a lawyer on 16 April 2026 after 57 of 63 citations were defective, and the SRA v. Kumar disciplinary tribunal striking a foreign lawyer off the register on 4 September 2026 for AI-fabricated authorities in his own defence. Iowa's new judicial rule, effective 2027, prohibits judges from delegating decision-making responsibility or deferring to AI-generated work product, while California's SB 574, in force 1 January 2027, requires disclosure of AI use, personal verification of every citation, and keeping confidential data out of public AI tools.

Where do we draw the line on AI in court filings? Scale There have been at least 2000 legal cases involving AI hallucinations. \ 1\ ref-1 Key rulings | Date | Case | What happened | Outcome | |---|---|---|---| | 12 Dec 2025 | Couvrette D. Or. <sup \ 2\ ref-2 </sup | 15 fake cases and 8 fake quotations, then concealment | $110,000 in fees and fines; the record total | | 16 Apr 2026 | Nebraska Supreme Court Lake <sup \ 3\ ref-3 </sup | 57 of 63 citations defective; first denied using AI | Suspended from practice | | 8 Jun 2026 | Withers v. City of Aberdeen N.D. Miss. <sup \ 4\ ref-4 </sup | Both sides filed hallucinated citations | All four lawyers removed; the drafters were barred from the district for 2 years; non-drafting co-signers fined $1,000 and forced to undertake a CLE. | | 4 Sep 2026 | SRA v. Kumar SDT, England <sup \ 5\ ref-5 </sup | AI-fabricated authorities in his own disciplinary defence | Struck off the register of foreign lawyers | | 30 Sep 2026 | State v. Horcasitas Ariz. Ct. App. <sup \ 6\ ref-6 </sup | AI-generated video of the dead victim “speaking” at sentencing | Sentence vacated as unreliable | Some observations follow: 1. Co-signing counts as using. In Withers, two lawyers who said they never use AI were still removed and fined. 2. Courts are tougher on lawyers than on pro se litigants. Pro se litigants mostly get fee orders and filing restrictions rather than penalties. The bench and the government | Date | Who | What | |---|---|---| | Oct 2025 | Judges Wingate S.D. Miss. and Neals D.N.J. <sup \ 7\ ref-7 </sup | Withdrew orders after staff used Perplexity and ChatGPT. Disclosed only after Senate questions. | | Jul 2026 | Chief Judge Jarbou W.D. Mich. <sup \ 8\ ref-8 </sup | Rebuked the Justice Department for an apparently AI-generated Sixth Circuit citation, without sanctioning it. | | 19 Aug 2026 | Judge Cooper D.D.C. <sup \ 9\ ref-9 </sup | Enjoined HHS grant changes resting on studies that do not exist. | One response to the above is Iowa’s new judicial rule 2027 , which explicitly prohibits judges from delegating decision-making responsibility or deferring to AI-generated work product. \ 10\ ref-10 Rules on paper | Instrument | In force | Requires | |---|---|---| | New York, 22 NYCRR Part 161 <sup \ 11\ ref-11 </sup | 1 Jun 2026 | AI permitted. Signing certifies a review for fabricated material. No disclosure. | | Florida, R. 2.515 + AOSC26-12 <sup \ 12\ ref-12 , \ 13\ ref-13 </sup | Jun 2026 | Signers certify that authorities exist and are accurately cited. Courts may not require AI disclosure. | | Connecticut, Indiana <sup \ 10\ ref-10 </sup | Jun–Aug 2026 | Verification or authenticity certification, without disclosure. | | California, SB 574 <sup \ 14\ ref-14 </sup | 1 Jan 2027 | Disclose AI use to the court. Personally read and verify every citation. Keep confidential data out of public AI. Arbitrators may not delegate decisions. | | Tenth Circuit, proposed <sup \ 15\ ref-15 </sup | Comments to 18 Oct 2026 | AI-assisted filings certify independent human verification. | | Proposed FRE 707 <sup \ 16\ ref-16 </sup | Revised May 2026 | AI output offered as evidence without an expert must meet Rule 702 reliability. | Florida first had a patchwork of circuit-level AI disclosure orders, then replaced them with one rule that never mentions AI. \ 13\ ref-13 The line | Zone | What falls in it | |---|---| | Settled: allowed | Drafting, editing, summarising and research leads with any tool, provided a human checks them. | | Contested | Whether to disclose AI use, whether client data may go into public tools, how hard to punish, how far to trust AI summaries of the record, and the courts own use of AI. | | Over the line | Citing an authority or quotation you have not read, at any level of intent. Inventing facts or testimony. Hiding prompts for AI readers. | Conclusion So what’s the verdict? - Using AI is fine; signing unread output is not. - Courts are converging on verification, not disclosure. - Sanctions have moved from fines to licences. - Candour sets the penalty. Some lawyers want LLMs out of legal work altogether. Others want much stronger standards, specifically for judges. Some see AI as an avenue towards wider access to justice. The overall consensus seems to be: you file it, you own it. As long as accountability is enforced, stripping AI use from the law completely would be an overreaction. References 1. Damien Charlotin, ‘AI Hallucination Cases Database’ last updated 5 October 2026 <www.damiencharlotin.com/hallucinations/ https://www.damiencharlotin.com/hallucinations/ 2. Amanda Robert, ‘Federal judge hands down $110K penalty against 2 lawyers for AI errors in court documents’ ABA Journal , 17 April 2026 <www.abajournal.com/news/article/oregon-federal-judge-hands-down-110000-penalty-for-ai-errors https://www.abajournal.com/news/article/oregon-federal-judge-hands-down-110000-penalty-for-ai-errors 3. Brian Mastre, ‘Nebraska Supreme Court suspends Omaha attorney over AI use’ 10/11 NOW , 16 April 2026 <www.1011now.com/2026/04/16/nebraska-supreme-court-suspends-omaha-attorney-over-ai-use/ https://www.1011now.com/2026/04/16/nebraska-supreme-court-suspends-omaha-attorney-over-ai-use/ 4. Amy O’Neal, ‘Judge Cancels Trial After Both Sides Submitted Filings With AI Hallucinations’ FindLaw , 16 June 2026 <www.findlaw.com/legalblogs/technologist/judge-cancels-trial-after-both-sides-submitted-filings-with-ai-hallucinations/ https://www.findlaw.com/legalblogs/technologist/judge-cancels-trial-after-both-sides-submitted-filings-with-ai-hallucinations/ 5. Kennedys, ‘SDT strikes off lawyer over AI-generated false authorities’ 1 October 2026 <www.kennedyslaw.com/en/thought-leadership/case-review/2026/sdt-strikes-off-lawyer-over-ai-generated-false-authorities/ https://www.kennedyslaw.com/en/thought-leadership/case-review/2026/sdt-strikes-off-lawyer-over-ai-generated-false-authorities/ 6. ‘Arizona manslaughter sentencing vacated due to use of AI “victim impact statement”’ FOX 10 Phoenix , 30 September 2026 <fox10phoenix.com/news/arizona-manslaughter-sentencing-vacated-due-use-ai-victim-impact-statement https://fox10phoenix.com/news/arizona-manslaughter-sentencing-vacated-due-use-ai-victim-impact-statement 7. Debra Cassens Weiss, ‘2 federal judges reveal AI use by staff members led to error-riddled opinions’ ABA Journal , 24 October 2025 <www.abajournal.com/news/article/2-federal-judges-reveal-ai-use-by-staff-members-led-to-error-riddled-opinions https://www.abajournal.com/news/article/2-federal-judges-reveal-ai-use-by-staff-members-led-to-error-riddled-opinions 8. ‘DOJ Called Out by Judge for Alleged AI Use, Fake Case Citation’ Bloomberg Law , 16 July 2026 <news.bloomberglaw.com/litigation/doj-called-out-by-judge-for-alleged-ai-use-fake-case-citation https://news.bloomberglaw.com/litigation/doj-called-out-by-judge-for-alleged-ai-use-fake-case-citation 9. ‘Court Tells HHS To Stop Using AI To Cite Fake Studies, Or Willfully Misinterpret Others In Grant Solicitations’ Above the Law , 4 September 2026 <abovethelaw.com/2026/09/court-tells-hhs-to-stop-using-ai-to-cite-fake-studies-or-willfully-misinterpret-others-in-grant-solicitations/ https://abovethelaw.com/2026/09/court-tells-hhs-to-stop-using-ai-to-cite-fake-studies-or-willfully-misinterpret-others-in-grant-solicitations/ 10. ‘From AI Disclosure to Citation Accuracy: Legal AI Governance, April to September 2026’ Legal AI Governance , 1 October 2026 <legalaigovernance.com/blog/legal-ai-governance-april-september-2026/ https://legalaigovernance.com/blog/legal-ai-governance-april-september-2026/ 11. 22 NYCRR pt 161, ‘Use of Artificial Intelligence Technology’ New York State Unified Court System, effective 1 June 2026 <www.nycourts.gov/rules/part-161-use-artificial-intelligence-technology https://www.nycourts.gov/rules/part-161-use-artificial-intelligence-technology 12. Jones Day, ‘Supreme Court of Florida Creates Statewide Standard for AI Certification’ 17 June 2026 <www.jonesday.com/en/insights/2026/06/supreme-court-of-florida-creates-statewide-standard-for-artificial-intelligence-certification-in-court-filings https://www.jonesday.com/en/insights/2026/06/supreme-court-of-florida-creates-statewide-standard-for-artificial-intelligence-certification-in-court-filings 13. In re Representations by Signers of Filings , Administrative Order AOSC26-12 Supreme Court of Florida, 28 May 2026 <flcourts-media.flcourts.gov/content/download/2489379/29182858 https://flcourts-media.flcourts.gov/content/download/2489379/29182858 14. Philippa J Balestrieri and others, ‘California Enacts Rules Governing Lawyers’ Use of Generative AI’ Holland & Knight , 1 October 2026 <www.hklaw.com/en/insights/publications/2026/10/california-enacts-rules-governing-lawyers-use-of-generative-ai https://www.hklaw.com/en/insights/publications/2026/10/california-enacts-rules-governing-lawyers-use-of-generative-ai 15. Kit Yona, ‘10th Circuit Court of Appeals Proposes Rule To Require Certification of Filings Using Artificial Intelligence’ FindLaw , 25 September 2026 <www.findlaw.com/legalblogs/technologist/10th-circuit-court-of-appeals-proposes-rule-to-require-certification-of-filings-using-artificial-intelligence/ https://www.findlaw.com/legalblogs/technologist/10th-circuit-court-of-appeals-proposes-rule-to-require-certification-of-filings-using-artificial-intelligence/ 16. Derek T Luke and Shane D Valenzi, ‘Proposed Federal Rule of Evidence 707: What Business and Their Lawyers Need to Know’ Eckert Seamans , 27 May 2026 <www.eckertseamans.com/legal-updates/proposed-federal-rule-of-evidence-707-what-business-and-their-lawyers-need-to-know https://www.eckertseamans.com/legal-updates/proposed-federal-rule-of-evidence-707-what-business-and-their-lawyers-need-to-know