cd /news/ai-tools/pdf-to-markdown-for-llm-pipelines-in… · home › topics › ai-tools › article
[ARTICLE · art-170630] src=dev.to ↗ pub= topic=ai-tools verified=true sentiment=↑ positive

PDF to Markdown for LLM pipelines in .NET – without Python and without native libraries

A developer released miPDFlib, an Apache-2.0 PDF library written in pure managed .NET code, which analyzes page layout to export structured Markdown and heading-based chunks for retrieval-augmented generation pipelines without Python or native dependencies. The library's PdfLayoutExport.ToMarkdown detects columns, headings, tables, headers and footers, avoiding the interleaved text and lost table structure that naive line-by-line extraction produces. The author notes that layout-aware extraction keeps tables and headings intact and yields cleaner chunks for embedding.

by read4 min views3 publishedNov 16, 2026

Most retrieval-augmented generation (RAG) pipelines start the same way: take a PDF, pull out the text, split it into

chunks, embed the chunks. The first step looks trivial and is where many pipelines quietly lose quality. A language

model can only answer from what the extractor hands it – and a typical extractor hands it text in the order the

characters happen to sit on the page.

This article shows what goes wrong, and how to get structured Markdown and clean chunks out of PDFs in a .NET service

with miPDFlib, an Apache-2.0 PDF library in pure managed code. (Disclosure: I

wrote it.)

Take an ordinary report page: a heading, two columns of text, a table, a running head at the top and a page number at

the bottom. Read line by line, as most extractors do by default, it comes out like this:

Water Utility – Quarterly Report
2 Operations
Pumps in the northern plant ran at Energy use per cubic metre fell
92 percent of their rated capacity for the fourth quarter in a row. The
during the second quarter. The two largest saving came from the new
older units were replaced in May, variable-speed drives, which adjust
which cut maintenance hours by the pump speed to the demand in
almost a third. the network.
Plant Capacity Energy (kWh/m³)
North 92 % 0.41
South 87 % 0.45
Page 1

Every word is there, so a quick check passes. But the sentences of the two columns are interleaved ("ran at Energy use

per cubic metre fell"), the table has lost its columns, and the running head and page number sit in the middle of the

content – on every page. Chunk this and embed it, and the retriever finds chunks that mix two topics, the model reads

numbers without knowing which column they belong to, and "Page 1" ends up in the vector index a few hundred times.

The structure is there on the page; it just has to be read. PdfLayoutExport.ToMarkdown analyses the layout first –

columns, blocks, headings, lists, tables, headers and footers – and writes what it finds:

using MiPdf;
using MiPdf.Text;

using var document = PdfDocument.Open("report.pdf");
string markdown = PdfLayoutExport.ToMarkdown(document);

Pumps in the northern plant ran at 92 percent of their rated capacity during the second quarter. The two older units were replaced in May, which cut maintenance hours by almost a third.

Energy use per cubic metre fell for the fourth quarter in a row. The largest saving came from the new variable-speed drives, which adjust the pump speed to the demand in the network.

| Plant | Capacity | Energy (kWh/m³) |
| --- | --- | --- |
| North | 92 % | 0.41 |
| South | 87 % | 0.45 |

## Appendix

What happened on the way:

Markdown is a good intermediate format for language models: headings and tables survive, and it costs far fewer

tokens than HTML.

For retrieval you usually do not want one long Markdown string but chunks that each belong to one section. The layout

analysis behind the export is available directly, so chunking by heading takes a few lines:

var chunks = new List<(string Heading, string Text)>();
string heading = "";
var text = new System.Text.StringBuilder();

foreach (var page in PdfLayoutAnalyzer.Analyze(document))
    foreach (var block in page.Blocks)
    {
        switch (block.Kind)
        {
            case LayoutBlockKind.Header or LayoutBlockKind.Footer:
                continue;                                         // running heads and page numbers
            case LayoutBlockKind.Heading:
                if (text.Length > 0) chunks.Add((heading, text.ToString().Trim()));
                heading = block.Text;
                text.Clear();
                break;
            case LayoutBlockKind.Table:
                foreach (var row in block.Table!.Cells.GroupBy(c => c.Row))
                    text.AppendLine(string.Join(" | ", row.Select(c => c.Text)));
                break;
            default:
                text.AppendLine(block.Text.ReplaceLineEndings(" "));
                break;
        }
    }
if (text.Length > 0) chunks.Add((heading, text.ToString().Trim()));

For the sample report this gives one chunk under "2 Operations" with both paragraphs and the table rows – the heading

can go into the chunk's metadata or be prepended to the text before embedding. Each block also has its Bounds, the

page number and its lines with word positions, so an answer can point back to the exact place in the PDF.

Layout analysis is heuristic, so I measured it rather than trusting a few examples:

The same analysis also exports to hOCR, ALTO and PAGE XML, if your pipeline or an archive expects those formats.

Better to know them before relying on it:

In Python, tools such as Docling or pymupdf4llm do this job, and they are good at it. For a .NET service, though, a

Python sidecar means a second runtime, a second deployment and a process boundary – and PyMuPDF is AGPL or commercial.

miPDFlib runs in-process, has no native dependencies (so it works the same on Windows, Linux and in containers), and is Apache-2.0, so it can go into closed commercial products. The same library also creates, fills,

signs and archives PDFs, which helps when the documents that come out of the pipeline need to go back into a PDF.

dotnet add package miPDFlib

The text extraction guide covers the

options (columns, Markdown export settings, letters with fonts and colors, vector paths). If a PDF of yours comes out

wrong, an issue with the file – or a synthetic copy of its layout – is the fastest way to get it fixed.

── more in #ai-tools 4 stories · sorted by recency
── more on @mipdflib 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/pdf-to-markdown-for-…] indexed:0 read:4min 2026-11-16 · —