Exit codes lie when PDF extraction yields nothing

Claude Codecheerfully told me the document was empty. The PDF wasn't empty — it was a browser screenshot export with zero text layer. Every tool in the chain did exactly what it was designed to do, reported success, and the stacked successes produced a lie I believed.

The real failure point #

markitdown isn't broken. It extracts embedded text and deliberately doesn't OCR — documented design decision. pdfminer

and PyMuPDF both report 0 characters. The converter exiting non-zero on empty documents would be wrong in a more annoying way.

The bug lives in every integration that treats exit code as evidence of yield. Exit code answers "did the process complete?" I read it as "did we get the text?" Different questions, different answers for scanned pages.

$ markitdown screenshot.pdf -o out.md
$ echo $?
0
$ wc -c out.md
0 out.md

My integration checked the return code, saw success, cached the result, handed the model a path to an empty file. The model read nothing and concluded the document was empty. Reasonable from its position.

Why byte thresholds fail #

The obvious fix — reject anything under 500 bytes — catches the screenshot. It misses the case that actually cost me time: a course completion certificate. One decorative graphic, one title line as real text. Conversion yields:

Certificate of Completion

Thirty-nine characters. Sails through emptiness checks, gets cached as successful, and the model reports your certificate says "Certificate of Completion" and nothing else.

Near-misses are the expensive failures: slide decks exported as page images with footers, contracts scanned at an angle with headers that OCR'd once. They all return some characters.

What actually works #

I've settled on a two-layer check that catches both failure modes:

1. Structural validation — does the output contain paragraph-like structures, not just isolated lines? A real document has sentences, line breaks, recurring patterns. A certificate has one line.

2. Density heuristic — characters per page. If a 10-page PDF yields 200 characters total, extraction failed regardless of exit code. Threshold varies by domain (legal > technical > certificates), but 50 chars/page is a starting baseline.

def extraction_quality(text: str, page_count: int) -> bool:
    if len(text) < 50 * page_count:
        return False
    paragraphs = [p for p in text.split('\n\n') if len(p.strip()) > 20]
    return len(paragraphs) >= max(2, page_count // 3)

3. Sample verification — feed the first 500 chars to a cheap model with a strict prompt: "Does this look like meaningful document content or extraction artifacts?" Costs pennies, catches the certificate case every time.

The pattern generalizes #

This isn't PDF-specific. Any pipeline where a transformer can succeed while producing useless output has this shape: OCR, speech-to-text, HTML-to-markdown, code transpilers. The fix is always the same — measure yield, not completion.

What's your threshold strategy? I've seen teams use word count, unique word ratio, even embedding distance from a "garbage" centroid. Curious what's worked in production.

Next Hard Gates Beat Long Prompts →

All Replies （3） #

pdftotext

returns 0 but outputs nothing unless you force OCR first