I Tried to Prove DocLang Beats Markdown for PDF→LLM. The Data Said Otherwise.

A developer's experiment comparing DocLang XML markup to Markdown for feeding PDFs to LLMs found that all three representations produced identical answers—100% accuracy on facts and 87.5% on structure questions—when the whole document was in context, with Markdown being fastest and cheapest per call. The test used a 15-page construction RFP parsed once into a DoclingDocument and serialized by docling-core's first-party exporters, with gpt-5.4-mini via Azure OpenAI as the model. DocLang's expected advantage in locating answers did not materialize, and its output lacked hierarchical structure due to the layout model's failure to recover exhibit groupings.

If you’re building anything that feeds PDFs to an LLM — RAG, document Q&A, contract review — you hit the same fork early: what text representation do you hand the model? The default answer is Markdown. Parse the PDF, export to markdown , stuff it in the context window. It's cheap, it's readable, every tokenizer likes it. The newer answer is DocLang https://www.doclang.ai/ — an XML-based, “AI-native” markup for unstructured documents. It keeps headings, lists, tables, reading order, and even bounding-box coordinates, all in a schema designed to be read directly by a language model. On paper it should dominate Markdown: same content, but structured . I went in wanting DocLang to win. I built the experiment to show it. Here’s what actually happened. Source document: a real “General Conditions and Fee Request for Proposal” — a 15-page construction RFP with a project-info section and six exhibits A–F . It’s a scanned-style PDF, the kind you actually get in the wild. The critical design choice — one parse, many renderings. My first attempt at this experiment had a fatal flaw: I compared my hand-written PDF→DocLang converter against Docling’s mature, years-tuned Markdown exporter. That’s not “Markdown vs DocLang.” That’s “my weekend project vs a library with hundreds of contributors.” So I rebuilt it. The PDF is parsed once into a DoclingDocument. Then every representation is produced by docling-core's own first-party serializers on that single shared object: No hand-written converter anywhere in the loop. Any difference between the arms is now attributable to the notation , not to converter maturity. The model: gpt-5.4-mini via Azure OpenAI, JSON-mode, whole document in the system prompt. The questions: two sets, each graded by an LLM judge against a gold answer and a gold location “which section/exhibit is this in?” . Everything is in the repo: doclang-impl/experiment/ — prepare inputs.py builds the arms, run ci.py runs the comparison. With the whole document in context, all three representations produced the same answers — 100% on facts, 87.5% on structure questions. The one structure-set miss “is there a substantive Exhibit G?” was shared by every arm: it’s a text-extraction gap, not a format gap. The notation did not change whether the model could find and state the answer . Not for flat facts, not for “which exhibit is this in,” not for cross-references. Here’s where it gets expensive: That’s a per-call cost you pay on every single query, for zero measured accuracy gain on this document. No surprise, but worth showing: more input tokens → slower first token. Markdown was fastest in every cell. The scatter makes it plain — the three arms form three vertical bands by token count, and latency climbs with them. This is the one I expected DocLang to win — “which section is this answer in?” is exactly what structure should help with. The per-question grid shows where the misses land: The answer squares are almost all green. The red dots — wrong cited location — cluster on “DocLang no locations .” Here’s the thing I didn’t expect. When I looked at what export to doclang actually produced: <doclang version="0.7" <heading level="2" <location value="53"/ <location value="30"/ <location value="228"/ <location value="39"/ Wind Point Partners WPP </heading <heading level="2" ... General Conditions and Fee Request for Proposal </heading <text ... November 21, 2025 </text Every heading is level="2". There are no <group containers wrapping the exhibits. The running page header "Wind Point Partners WPP " is repeated ~12 times as a real <heading element. This isn’t a serializer bug. Docling’s layout model parsed this scanned PDF and didn’t recover the exhibit hierarchy — so there’s no nested structure in the DoclingDocument for export to doclang to serialize. The DocLang output faithfully reflects a flat parse. Which means: on this document, DocLang is Markdown’s exact information content, wrapped in XML that costs 1.5–2.6× the tokens. The structural advantage I was counting on requires structure the parser never extracted. My first write-up compared two hand-rolled DocLang converters against Docling’s Markdown exporter and concluded “flat DocLang is worse, my structured version closes the gap.” That comparison was confounded — I was measuring my converter, not the format. Removing the confound, the conclusion held and got sharper: the official DocLang serializer is also flat for this PDF, ties on answers, and loses on cost and latency. The format didn’t fail. The parser didn’t give the format anything to work with. This experiment stress-tests exactly one regime: one small document, full context, single-doc Q&A. In that regime, Markdown wins — cheaper, faster, equal accuracy. Full stop. DocLang’s value is in the regimes I didn’t test: 1. When the parser actually recovers structure. A born-digital PDF with a real outline, or a pipeline that post-processes the DoclingDocument tree to rebuild sections before serializing. Then export to doclang carries <group nesting that Markdown flattens away. 2. Retrieval, when the corpus doesn’t fit in context. DocLang gives you addressable units — <group , label — as natural chunk boundaries. Split Markdown on and your running page headers bleed into every chunk. This is DocLang's home turf and it's completely untested here. 3. Round-trip fidelity. Tables as <tabular , formulas, forms as <field region , checkboxes, reading order, RTL — Markdown drops all of it. And if you need to cite "page 12, this bounding box" back to the source pixels, the <location elements are the entire point. That 2.6× token cost is buying you provenance. 4. Scale. ~46–156% overhead per call here. Across a retrieval pipeline with clean unit boundaries, that math can flip. I Tried to Prove DocLang Beats Markdown for PDF→LLM. The Data Said Otherwise. https://pub.towardsai.net/i-tried-to-prove-doclang-beats-markdown-for-pdf-llm-the-data-said-otherwise-19d256cbaa00 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.