# State Farm lawyers just admitted to using fake AI cases in court

> Source: <https://promptcube3.com/en/news/6793/>
> Published: 2026-08-18 13:33:35+00:00

# State Farm lawyers just admitted to using fake AI cases in court

For those of us into prompt engineering, this is a classic failure of grounding. When you ask an LLM to find a specific legal precedent, it often prioritizes the *pattern* of a legal citation over the *fact* of the case's existence. If the model can't find a perfect match, it "predicts" what a winning citation would look like based on the surrounding context.

To avoid this in a real-world AI workflow, you can't just rely on a single prompt. You need a [RAG](/en/tags/rag/) (Retrieval-Augmented Generation) setup where the AI is forced to pull from a verified index of legal documents before synthesizing an answer. If you're building a tool for professional use, a "citation verification" step is non-negotiable.

Here is a basic logic flow for a verification agent that could have prevented this mistake:

``` python
def verify_citation(generated_text, legal_database):
    citations = extract_citations(generated_text)
    verified_citations = []
    
    for cite in citations:
        if legal_database.exists(cite):
            verified_citations.append(cite)
        else:
            # Flag as hallucination
            flag_for_human_review(cite)
            
    return verified_citations
```

This is a huge wake-up call for the "AI-first" movement in law. The efficiency gains of using an LLM to draft a brief are completely wiped out if you spend the next three months defending why you cited a non-existent court ruling. It proves that the human-in-the-loop isn't just a luxury; it's the only thing keeping the process credible.

If you're designing a deployment for any high-stakes industry, you have to assume the model will lie to you to sound more confident. The goal shouldn't be to find a model that never hallucinates—because that doesn't exist—but to build a system that catches the lie before it hits a judge's desk. Using a deep dive approach to validate every single external reference is the only way to ensure a professional output.

[Next Schemagic makes JSON Schema actually readable for non-coders →](/en/news/6789/)