Introducing Precision Mode in AI Extract: achieve frontier accuracy on long documents and complex schemas where existing approaches fall short.
by Jane Zhang, Arnav Singhvi, Ivan Zhou, Archika Dogra, Matthew Ding and Nihit Desai
Every enterprise has valuable data trapped in messy, unstructured documents. Today, Databricks Document Intelligence helps thousands of customers put their data to work, turning billions of pages into structured data that powers production pipelines, agents, and applications. Customers like Panasonic, EY-Parthenon, and Intercontinental Exchange (NYSE) use Document Intelligence on their most demanding workflows, processing millions of documents weekly.
When working with customers, we noticed several difficult extraction problems where existing large language model (LLM) or rules-based document extraction solutions fall short:
Today, we’re excited to introduce Precision Mode in our document extraction API, ai_extract, setting a new bar for accuracy on the most complex enterprise documents and tasks.
Precision Mode combines our custom-trained models for document extraction with an agentic harness to deliver reliable and accurate extraction on long documents, large outputs, and reasoning-heavy schemas. Across benchmarks spanning roughly 9,000 complex documents, Precision Mode achieves the state of the art quality, outperforming the latest frontier models on extract accuracy by a large margin.
"At Intercontinental Exchange, we process millions of complex, highly variable financial documents every month. Document Intelligence helps us turn that complexity into structured market intelligence, enabling us to move faster, deliver greater value to our clients, and unlock agentic workflows that accelerate analysis and decision-making at scale."—Anand Pradhan, CTO and Head of AI, Mortgage Data at Intercontinental Exchange (NYSE)
To push accuracy on the hardest extraction tasks, our research and engineering teams approached document extraction quality from two layers: customizing the model itself and building an effective agent harness around the model.
To validate Precision Mode, we designed our evaluation benchmarks around workloads that push existing approaches to their limits.
Concretely, we evaluated Precision Mode on roughly 9,000 documents spanning the three extraction challenges it was designed to solve. The evaluation includes documents up to 2,000 pages, invoices with thousands of line items, dense multi-page tables and charts, schemas with more than 300 deeply nested fields, and reasoning-heavy tasks that require cross-referencing information across a document.
The documents come from two sets of benchmarks:
Together, these datasets cover documents including 10-K filings, bills of lading, technical manuals, financial documents, clinical notes, government patent and funding applications, and more.
A natural starting point for document extraction is a single frontier-model call: pass in the document and schema, and ask the model to return the structured output. But on the dense and complex workloads we evaluate, that approach quickly breaks down. Long documents can exceed model context limits, causing inaccurate and incomplete results.
So we benchmarked against a stronger, more realistic baseline: chunk-and-merge. We split each document into smaller chunks, extract from each independently, and merge the results into a final output—the same pattern we see engineers use when a single model call isn't enough.
We tested the chunk-and-merge approach using leading GPT, Claude, and Gemini models with their default API settings. Then, we compared each against Precision Mode on extraction accuracy. ¹
Across our benchmarks, Precision Mode reaches 94.7% accuracy, outperforming the strongest frontier model chunk-and-merge baseline, GPT-5.6 Sol, by seven points.
Particularly, on difficult long-document workloads, we observed frontier-models encounter numerous operational failure modes including chunk timeouts, truncated outputs, and incomplete final merges that did not conform to the requested schema. On the other hand, Precision Mode’s agentic approach is robust against these failure modes, and our custom-trained extraction models keep extractions efficient and accurate.
For your most complex document extraction tasks, AI Extract Precision Mode is now available. Set the mode to precision
when calling ai_extract function, or turn on the precision mode toggle in the Information Extraction UI on the Agents page:
Footnotes:
¹ We define accuracy as the fraction of extracted objects that match the ground-truth object. Scoring depends on type. Primitives (booleans, floats, integers, enums) use direct match. Strings try direct match first, then fuzzy match, then an LLM judge. Arrays are scored by finding the closest pairing between predicted and expected items, then averaging across pairs. Objects are scored per field by type, then averaged across all fields.
Subscribe to our blog and get the latest posts delivered to your inbox.