{"slug": "filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and", "title": "Filevine Takes On Shepard’s and KeyCite With Its Own AI Case Law Citator and Hallucination Checker", "summary": "Filevine launched an AI-native citator and a companion brief-checking tool inside LOIS, its Legal Operating Intelligence System AI platform, claiming performance equal to or better than LexisNexis's Shepard's and Thomson Reuters's KeyCite. The citator verifies whether a proposition is still good law at the passage level, while the anti-hallucination tool checks briefs for fabricated citations and altered quotations. CEO Ryan Anderson said the company delayed promotion until the product was fully ready, and the anti-hallucination tool is released for the first time this week.", "body_md": "The legal technology company [Filevine](https://www.filevine.com/) has launched an AI-native citator and a companion brief-checking tool inside LOIS, its Legal Operating Intelligence System AI platform, that its CEO says perform as well as or better than the leading legal citators from LexisNexis and Thomson Reuters.\n\nThe citator has two core features. One checks a filed or draft brief for hallucinated citations, altered quotations, and mischaracterized authority. The other checks whether a proposition is still good law by letting a lawyer highlight a specific passage in an opinion and find out whether later courts have undercut the holding expressed in the passage.\n\nAlthough Filevine had previously [announced the citator in June](https://www.filevine.com/news/filevine-disrupts-legal-research-duopoly-with-ai-built-to-verify-exact-legal-passages-not-just-cases/) as part of its new LOIS Legal Research offering, which provides access to millions of opinions from U.S. federal and state courts, CEO Ryan Anderson, in a briefing this week, told me the company had not heavily promoted the citator until it believed the product was fully ready for customers to use.\n\nThe company had not previously announced the anti-hallucination tool, which is being released for the first time this week.\n\nAs Filevine described the citator in its June release: “While legal verification tools rely on explicit citation relationships and apply treatment signals to entire opinions, LOIS Legal Research operates at the user-selected passage level.” This enables legal professionals to verify the exact rule they plan to rely on in their argument.\n\n**The Challenge Of Citators**\n\nI have previously written that developing a strong citator tool [is the Holy Grail of a legal research platform](https://www.lawnext.com/2024/07/new-citators-from-vlex-and-paxton-underscore-that-they-are-the-holy-grail-for-legal-research-companies.html). Among companies to have tried it in recent years were [vLex and Paxton](https://www.lawnext.com/2024/07/new-citators-from-vlex-and-paxton-underscore-that-they-are-the-holy-grail-for-legal-research-companies.html), [Descrybe](https://www.lawnext.com/2025/06/free-legal-research-site-descrybe-ai-launches-a-paid-suite-of-legal-research-tools-including-its-own-citator.html), and [Midpage](https://www.lawnext.com/2025/02/legal-research-startup-midpage-passes-threshold-as-it-achieves-full-case-coverage-and-readies-citator-launch.html).\n\nBut it is not an easy task. In fact, in my briefing with Filevine this week, [John Rizner](https://www.linkedin.com/in/john-r-278b6b2a8/), the product manager who led the development of the citator, pointed out something that most law librarians will know but that may be news to almost everyone else, which is that the two citators lawyers have long considered to be the gold standard routinely disagree with each other about whether a case is still good law.\n\nHe pointed to a [2018 study in the Law Library Journal](https://scholarship.law.wm.edu/libpubs/131/) by Paul Hellyer, a librarian at William & Mary Law School, who reviewed 357 citing relationships that at least one of Shepard’s, KeyCite or BCite had labeled negative.\n\nAll three citators agreed that the treatment was negative only 53 times, meaning that in 85% of the relationships in Hellyer’s sample, the three citators did not agree that a negative treatment had occurred.\n\nHellyer also examined whether the flags that did appear were correct. Counting relationships the citators omitted or mislabeled as positive or neutral, together with those labeled negative but incorrectly described, he found overall failure rates of 33% for Shepard’s, 38% for KeyCite and 72% for BCite.\n\nRizner also cited a second line of research, from Susan Nevelow Mart, the former director of the law library at the University of Colorado, whose [2013 examinations of digest and citator results](https://scholar.law.colorado.edu/faculty-articles/102/) found that the overlap between the Westlaw and Lexis citator systems on relevant results was small – only about a third.\n\nAs the abstract of Mart’s article put it: “Neither algorithm is doing a very good job of finding all the relevant results; the overlap between the two citator systems is not that large.”\n\nAhead of our briefing, Rizner ran his own test. He pulled *Freedom Mortgage Corp. v. Engel* on both platforms and filtered each down to negative treatment in reported opinions. Westlaw found 28 and Lexis found 13. The overlap between the two sets, by his count, was about 41%.\n\n“If my law firm happens to have Lexis, I might be missing opinions because I don’t have Westlaw, or vice versa,” Rizner said.\n\nWhat that means, Anderson said, is that even though the legacy citators have been best in class up until now, that does not mean that they do not make mistakes.\n\n**Citators As An Essential Tool**\n\nAt this point, you may be thinking: “Sure, but what does this all have to do with Filevine, which is not a legal research platform?”\n\nIn my opinion, it has everything to do with Filevine and every other legal platform in which legal professionals are using AI to draft legal documents that rely on legal citations.\n\nThanks to concerns about AI accuracy and [rampant hallucinations](https://www.damiencharlotin.com/hallucinations/), the role – or maybe I should say the place – of the citator is evolving.\n\nFor most of their history, citators were legal research products – the last step before you wrapped up your research. But, as Anderson argued in our briefing, citation validation is now a required step in any legal workflow that produces a document relying on case law.\n\n“Usually you’re taking facts, applying them to law and drafting,” he said. “That is the day-to-day shoe leather work of a lawyer.”\n\nFilevine already held the facts, he said, but the missing pieces were a case law corpus and a way to confirm that what the corpus returns is still good law.\n\n“And I think now there’s another step,” he said, “which is make sure nothing you’ve looked at is hallucinated.”\n\nEarlier this year, at a meeting of roughly 20 large firms that Filevine convened, Anderson heard the same anxiety about hallucinations expressed repeatedly. One litigator said his top-10 firm now requires three separate sign-offs before any filing goes out, with the briefing associate required to attest that the associate had checked every citation and every holding.\n\n**Filevine’s Answer to This Need**\n\nSo what is Filevine’s answer to this need? In our briefing this week, Anderson and Rizner demonstrated how the brief auditor and citator work.\n\nFor the brief auditor, after It ingests a document, it extracts and analyzes every citation, and then reports four things in a side-by-side chart:\n\n- The proposition the brief says the authority supports.\n- The citation given.\n- What the system found in the opinion.\n- The status, which can be verified, unverified, suspect, ambiguous, mischaracterized or wrong case name.\n\nUsing briefs pulled from PACER in matters where courts had already found problems, Anderson showed me examples. In one, a brief asserted that the 11th Circuit had squarely held that “100% healed” policies violate the ADA. The tool reported that the phrase appears in the opinion only in the plaintiff’s own request to return to full duty.\n\nIn another, drawn from *Knight v. Baptist Hospital of Miami*, the brief cited a passage for the proposition that disparate discipline of similarly situated employees is evidence of pretext. The tool flagged that the quoted sentence does not appear in the opinion and that the court’s reasoning runs the other way, having found the comparator not similarly situated.\n\n“It’s really quite remarkable how subtle these hallucinations can be even when the case itself is not hallucinated,” he said. He noted that fabricated support often turns up in a dissent or in a recitation of case history rather than in the holding.\n\n**A Different Kind Of Citator**\n\nThe citator works differently from a traditional one. Rather than starting from a case and examining its citation graph, the user highlights the specific language at issue.\n\nFilevine then runs two passes – a traversal of the citation graph and a semantic similarity search – across its corpus for opinions using comparable reasoning. Opinions that come up as likely candidates are handed to LLMs, which distill the conflicts.\n\nThe output is a report keyed to the highlighted passage, with a severity rating and a written treatment for each authority.\n\nIn the results, a second tab generates what Rizner described as a synthetic treatise – a narrative discussion of the doctrinal tension, structured like a secondary source but built around the specific issue the user selected.\n\nBecause the semantic pass does not depend on one court citing another, Rizner said, it surfaces *sub silentio* conflicts, such as, for example, a 1985 decision and a 2005 decision that squarely contradict each other but never cite each other. This would be invisible to any citator built on a citation graph alone.\n\n**Evaluating the Results**\n\nFilevine gave me access to an as-yet unpublished paper evaluating its anti-hallucination tool.\n\nAccording to the paper, Filevine’s evaluation team started with a database of 1,314 actual matters in which a judge had identified potentially hallucinated citations tied to admitted or suspected use of gen AI. It narrowed that pool to federal filings and then to filings submitted by attorneys rather than pro se litigants. That left 103 cases, of which 68 had records available for full analysis.\n\nFor each of the 68 matters, Filevine pulled the court’s order and the offending brief, ran the brief through LOIS using an identical prompt, and compared the output against the order. LOIS worked only from its own case law corpus, blocking general web search to keep it from finding news coverage or blog abouts about the cases.\n\nAcross 2,073 citations, LOIS verified 61.8%, flagged 38.2% for attorney review, and marked 8.4% as severe, meaning they were mischaracterized holdings or citations pointing to the wrong or a non-existent case. Every document in the sample produced at least one flag. Among the highest-citation filings, the cleanest still had roughly 9% of its citations flagged, while the worst had 76%.\n\nSpecifically, LOIS was instructed to look for three categories of hallucinated cases: fabricated case law, false or misattributed quotations, and mischaracterized authority.\n\nFor fabricated case law, LOIS found them in multiple matters. The report gave the example of *Renschler v. Yosh*, 909 F. Supp. 2d 1197, a citation LOIS flagged because no case by that name appeared anywhere in the reporter database.\n\nFor altered quotations, it also found multiple examples. The paper points to a Rule 37 sanctions argument that quoted the 2nd Circuit case of *Shcherbakovskiy v. Da Capo Al Fine, Ltd.* as requiring fault “of the sanctioned party,” when the court actually wrote “of the deponent.”\n\nFilevine said the third category, mischaracterized authority, yielded “the exercise’s most critical observation,” which was the high number of instances when the citation appeared legitimate, but it did not stand for the proposition asserted, did not contain the argument attributed to it, or contained citations that LOIS could not access to confirm the authority.\n\nIn one filing, a brief cited *Watson v. State of California* for the proposition that California Government Code § 845.6 imposes an affirmative duty to summon medical care, when the opinion holds the statute confers broad immunity and the *Watson* plaintiff lost.\n\nIn another, *Nai v. National Asset Mortgage, LLC,* in the Western District of Michigan, counsel quoted RESPA’s limitations provision, 12 U.S.C. § 2614, as running from the “first” occurrence of the violation. The word “first” is not in the statute, meaning its insertion converted an occurrence-based trigger into a first-occurrence trigger, which counsel used to argue two tax years of claims were time-barred.\n\nNotably, the report says, in matters where the bad citation had already been flagged by a court, LOIS caught the same defect, and it identified authorities cited for propositions that contradicted the underlying argument.\n\n**A Few Other Notes**\n\nAs I noted above, the citator bases its analysis on the text highlighted by the user. Given that, I asked Rizner whether it was possible that a lawyer who validates one holding in a long, multi-issue opinion would not necessarily learn that the case was reversed on an unrelated holding.\n\nHe confirmed that is possible, though he said the system’s severity ratings are designed to explain when a citing case engaged with the opinion but not with the issue at hand. Anderson suggested that a direct query, or highlighting the full opinion, would likely catch it, but acknowledged the company had not tested that.\n\nAs for the source of Filevine’s case law corpus, it is using the CourtListener library, supplemented by licensed material that provides reporter citations and pagination for cases where they are lacking from the CourtListener collection.\n\nThe collection does not currently include statutes or regulations, although Filevine plans to add them in the future.\n\n**Bottom Line**\n\nI have not yet directly tested Filevine’s citator and anti-hallucination tools, although I hope to do that soon.\n\nHowever, I think that this will prove to be of significant value to Filevine’s customers, many of whom are law firms with significant litigation practices.\n\nBy integrating within its AI platform both a corpus of case law and the ability to check citations against that corpus, Filevine is providing customers with an important safety check against hallucinations.\n\nOf course, as Anderson pointed out, none of that replaces the need for lawyers and legal professionals to validate their own work.\n\n“I don’t think it completely solves the problem,” he said. “A human still has to take a look. But at least we’re going to point you in the right direction.”", "url": "https://wpnews.pro/news/filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and", "canonical_source": "https://www.lawnext.com/2026/09/filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and-hallucination-checker.html", "published_at": "2026-09-02 14:24:33+00:00", "updated_at": "2026-09-02 14:54:08.527376+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-products", "ai-tools", "natural-language-processing"], "entities": ["Filevine", "LOIS", "Ryan Anderson", "John Rizner", "LexisNexis", "Thomson Reuters", "Shepard's", "KeyCite"], "alternates": {"html": "https://wpnews.pro/news/filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and", "markdown": "https://wpnews.pro/news/filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and.md", "text": "https://wpnews.pro/news/filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and.txt", "jsonld": "https://wpnews.pro/news/filevine-takes-on-shepards-and-keycite-with-its-own-ai-case-law-citator-and.jsonld"}}