Many bad RAG answers over PDFs start at extraction, not embeddings. I’ve been working on PDF to Markdown conversion and wanted to share what made the biggest difference, plus a small runnable example.
What breaks with plain text extraction
What worked (pdfplumber + pdfminer.six, both MIT)
Then chunk on headings: each chunk keeps its section title, which helps retrieval a lot.
Limits of this approach: no OCR (it only reads the existing text layer), and borderless tables still come out as text.
Full write-up with a runnable script that builds a two-column sample PDF, shows the broken output, then the fixed Markdown and heading-based chunks:
Disclosure: I also maintain a hosted version of this as a paid Apify Actor (mentioned at the end of the article), but everything above works with the open-source libraries alone.
I’d like to hear what others use. Have Docling, Marker or pymupdf4llm handled multi-column and CJK documents well for you, and how do you deal with borderless tables?