DocLang: a markup language for LLMs

IBM Research introduced DocLang, a constrained XML sub-language designed to make document structure more interpretable for large language models, complementing its open-source Docling parser. Docling, IBM's most successful open-source project with 32 million downloads and 64,000 GitHub stars, converts unstructured data from formats like PDF, PPT, XLSX, and MP4 into DocLang's byte-identical representation, aiming to reduce token usage and lower latency and inferencing costs. Lead researcher Peter Staar said DocLang's one-to-one mapping between semantic tokens and LLM tokens minimizes the model's parsing effort.

Introducing DocLang, a markup language for AI We spoke with Peter Staar, the lead researcher behind IBM’s popular document parser, Docling, about why generative AI needs its own document standard. AI is upending how we live and work, but the data that could really transform how businesses run is often locked away in outdated or proprietary formats. Docling, IBM’s open-source document parser, has helped to make this information more accessible, allowing enterprises to lift new insights from old charts, customer files, and annual reports. With 32 million downloads and 64,000 stars on GitHub, Docling is IBM’s most successful open-source project ever. It’s been integrated into everything from LangChain to Red Hat’s OpenShift, and even IBM’s own Granite family of LLMs https://www.ibm.com/granite . Now, the researchers behind Docling https://research.ibm.com/blog/docling-generative-AI are introducing its complement: DocLang https://github.com/doclang-project/doclang/blob/main/spec.md motivation , a sub-language of XML that has the potential to lower latency and inferencing costs by telling an AI model in fewer tokens where things like a headline ends and a chart caption begins. XML stands for Extensible Markup Language https://www.ibm.com/docs/en/i/7.5.0?topic=toolkit-xml-introduction , which was designed to store and move data across apps, databases, and web services. In the 1990s, front-end webpages moved to HTML because it was more forgiving of errors than XML, with its strict parsing rules. HTML also had fixed, ready-to-use visual tags making it easier to use. Though HTML became the markup language of the web, XML never really went away. Docling converts unstructured data from various formats PDF, PPT, XLSX, MP4 , into an internal representation that is serialized into DocLang, a constrained dialect of XML. DocLang’s flavor of XML was designed to be more interpretable for an LLM. We recently caught up with Peter Staar, the lead researcher behind Docling, to learn more about DocLang’s back story, its benefits, and what it will take for enterprises and others to adopt it. Why is something like DocLang needed? The document formats we use today were designed for different purposes. PDF tells a renderer where to put pixels. DOCX tells a word processor how to lay out a page. Neither were built for a machine that needs to understand the content. When you push these formats through an AI pipeline, reading order becomes ambiguous, tables collapse, figures lose their captions, and metadata disappears. All this happens before the model even sees the content, which caps its accuracy. How is DocLang different? Its only job is machine understanding. It's an open standard — think JSON for data, or HTML for the web — that natively encodes semantics, layout, bounding boxes, and reading order. Instead of trying to recover structure later, we define a representation from the start. We also take the position that a given piece of content should have exactly one encoding. Two systems that read a document correctly should produce byte-identical output. That property — one document, one representation — is something no previous standard guarantees, and it's what makes DocLang trainable and verifiable. Why did you choose XML as the basis for DocLang? We chose XML for its clean semantic-token link. Each semantic token has an explicit opening and closing form — <text and </text — and maps directly to a single LLM token. An open tag is one token, the close tag is another. That one-to-one correspondence means the model isn't wasting capacity learning to parse a fuzzy markup dialect. The structure of both the document and the token stream are identical. We cap the syntax vocabulary at about 1,000 tokens and push property semantics into nested elements rather than attributes, precisely to keep that mapping tight and minimize tokens. Where did you get the idea for DocLang? It came out of an observation while working on table recognition. We were training image-to-sequence models to read tables and output them in HTML. It worked, but it was inefficient and the models kept producing syntactically broken HTML. That led to Optimized Table Structure Language https://research.ibm.com/publications/optimized-table-tokenization-for-table-structure-recognition OTSL , which we published in 2023. Our insight was simple but powerful: if you design a minimal, purpose-built vocabulary instead of borrowing HTML, the model gets more accurate. Inference time goes down and the output is always syntactically correct. The representation does the work we'd been asking the model to do. If this works for tables, we thought, why not for a whole page? That became DocTags, and then SmolDocling https://arxiv.org/abs/2503.11576 , a 256-million parameter vision–language model VLM that could convert a page with tables, formulas, code, and charts, in one pass and compete with models 27 times larger. Along the way, we learned what not to do. DocTags wasn't XML-compliant or complete — it just couldn't express some document constructs cleanly. DocLang builds on the core idea of a token-efficient, machine-native representation that is also XML-compliant, complete, and canonical. What are the benefits? A few, and they compound. I’m going to list them out: - Higher accuracy with smaller models. The representation does more of the work, as we saw with OTSL and SmolDocling. - No hallucinated structure. DocLang’s grammar makes invalid output unrepresentable, be it a non-rectangular table or unclosed element. Whole classes of errors simply can't occur. - Lossless and grounded. Full table grids, figure positions, reading order, and bounding boxes are all preserved for the model to process. - Consistency you can build on. One canonical encoding per document means you can train on it, diff it, cache it, and verify it. It’s a format you can trust. - Governance built in. Things like personally identifiable information flags, RAG permissions, and training constraints live in the document head, not in fragile sidecar files, so compliance metadata stays with the content. - It's not just for documents. DocLang’s primitives extend to audio transcripts, images, and video segments, which matters as pipelines go multimodal. - Open, with no lock-in. It's an open standard under the governance of the Linux Foundation https://www.linuxfoundation.org/press/lf-ai-data-foundation-launches-doclang-specification-working-group-to-advance-an-open-standard-for-ai-native-documents , so it can be implemented or consumed with any tool or pipeline. What are the challenges ahead? A format is only as valuable as the ecosystem that adopts it. A spec with no implementations is just a document. Adoption will make or break DocLang, which is why we launched it with real support. A second challenge is balancing completeness and consistency. The world of documents is endlessly messy, and each time you add expressiveness to cover an edge case, you risk creating a new way to represent something. Holding the line on "one document, one representation" while still covering real-world complexity is hard, ongoing design work. It's why we're still at version 0.x and treating minor versions as potentially breaking. We'd rather get the canonical form right than freeze it prematurely. Migration cost is a third challenge. Anyone with an existing pipeline has to invest in tooling and retraining to convert. We think the accuracy gains pay for it, but it's a real cost we shouldn't wave away. We also face governance risk in keeping the standard genuinely open and not letting it fragment into vendor-specific versions. That's a people-and-process problem as much as a technical one. What will it take for DocLang to be widely adopted? Standards win when they're easy to adopt and obviously better, not when they're mandated. So, our strategy has three parts. First, you make it free, open, and neutrally governed. DocLang is an open spec under foundation governance with multiple organizations behind it. No vendor lock-in. That's the price of entry for anyone to trust a format with their documents. Then you meet people where they are. This is the most important lever. We're not asking the world to rewrite its pipelines from scratch — we're making DocLang a native output of tools people already use. Docling and ABBYY's FineReader Engine, a software development kit for optical character recognition, can already emit DocLang. The moment your existing parser can produce the standard format, adoption stops being a migration project. You also have to relentlessly prove your case. OTSL and SmolDocling showed that a better representation isn't a matter of aesthetics, it directly improves model accuracy and cuts cost. Developers adopt formats that make their systems measurably better. We lead with that evidence. With enough independent implementations, DocLang becomes the default. It’s the thing your parser emits and your model consumes. Nobody decides to use JSON, they just do. Will DocLang eventually make Docling obsolete? Not at all. They complement each other. Docling is a tool that turns messy real-world files into structured output. DocLang is a standardized format for expressing that output. If anything, DocLang makes Docling more valuable, because now Docling's output is an open standard that any downstream system can consume and that other tools can produce. The whole point of a standard is that no single tool owns it. A document converter and the format it targets aren't in competition any more than a compiler and machine code spec are. Related posts - ReleasePeter Hess Bringing a common language to AI evaluation /blog/every-evaluation-ever NewsKim Martineau IBM is committing up to $50 million worth of quantum compute access for the US Genesis Mission, and more /blog/ibm-us-genesis-mission-quantum-ai News IBM open sources CodeAlchemy, a massive synthetic dataset of high-quality code /blog/code-alchemy-for-synthetic-code ReleaseKim Martineau