RAGpipeline. Usually, you're forced to choose between a heavy JVM dependency, a bloated Python ML stack, or sending private data to a cloud API. For air-gapped environments—like banking or healthcare—cloud parsers are a non-starter, and I personally can't stand the "JVM tax" or the nightmare of managing pip wheels just to extract text.
I needed a deterministic, single-binary solution that lives entirely on-machine. That's why I built DeepDoc in Rust.
Comparing the usual suspects #
If you're setting up a document ingestion workflow, you've probably looked at these:
Apache Tika: The industry standard, but requires a JVM. It's overkill for a Go or Rust service and adds massive overhead to your container images.Python Stacks (unstructured, Docling): Great for tutorials, but they often require down ML weights and native dependencies. They are too slow for cold starts and often non-deterministic.Cloud Parsers (LlamaParse): Fast to implement, but they charge per page and require up sensitive data to a third-party server.
The DeepDoc approach #
The goal was a static Rust binary with zero runtime dependencies. No network calls, no shell-outs, and no model downloads. It transforms documents into clean Markdown while keeping tables intact.
Here is a real-world example of the output:
$ deepdoc report.docx
Prepared by the Platform team. This memo covers delivery and headcount for the third quarter.
## Delivery
We shipped the ingestion rewrite and cut p95 latency by 38%.
The nightly batch now runs in under nine minutes.
## Headcount
| Team | Engineers | Open roles |
| -------- | --------- | ---------- |
| Platform | 9 | 2 |
| Search | 6 | 1 |
| Data | 4 | 0 |
Technical implementation #
The architecture is intentionally simple to ensure reliability. Every supported format is first parsed into a neutral Document
model containing headings, paragraphs, lists, tables, and metadata. Once the data is in this intermediate shape, a pure function serializes it into Markdown, JSON, or plain text.
By decoupling the format parsing from the output serialization, the logic remains maintainable. Whether it's a .docx
or a .pptx
, once it hits the internal model, the Markdown conversion follows a single, tested code path. This ensures that the RAG index remains reproducible—a critical requirement for any production LLM agent.
Next Voice Cloning: Quality vs. Length →