From PDF to Evidence: Building a Production-Ready RAG Pipeline

A developer outlined a production-oriented RAG pipeline for processing complex PDFs such as contracts, invoices, and reports, arguing that retrieval quality depends heavily on document-processing quality. The architecture chains OCR/document intelligence, layout analysis, page-aware structure, metadata-rich chunking, hybrid keyword-plus-vector retrieval, and LLM output that returns structured findings with page-level citations as evidence. The developer recommends evaluating retrieval separately from answer generation, using precision and other retrieval metrics rather than judging only the final LLM response.

RAG is often described as a simple architecture: Documents → Embeddings → Vector Database → LLM That diagram is useful for understanding the concept, but it is far from enough for a production document-processing system. When the source documents are PDFs containing tables, scanned pages, headings, footnotes, forms, and complex layouts, the real challenge is not simply retrieving similar text. The real challenge is: Can the system generate an answer that can be traced back to the correct evidence in the original document? This article describes the architecture and engineering considerations behind a production-oriented RAG pipeline for document processing. Consider a system where users upload contracts, invoices, reports, or other business documents. A typical workflow looks like this: PDF ↓ Document Intelligence / OCR ↓ Layout Analysis ↓ Page-aware Document Structure ↓ Chunking + Metadata ↓ Search Index ↓ Hybrid Retrieval ↓ LLM ↓ Structured Finding ↓ Evidence + Citation The important observation is that retrieval quality depends heavily on document processing quality . If the original PDF is poorly converted into text, even the best embedding model cannot recover information that was lost during extraction. A PDF is not necessarily a collection of plain text. It can contain: Simply extracting all text and splitting it every 500 tokens can destroy important relationships. For example: Invoice Amount: ¥12,500,000 Tax: ¥1,250,000 Total: ¥13,750,000 If these values are separated incorrectly during chunking, the LLM may retrieve only part of the information. Therefore, the ingestion pipeline should preserve document structure whenever possible. One of the most important pieces of metadata in a document RAG system is the source page . Instead of storing only: { "text": "The contract amount is..." } I prefer a structure closer to: { "text": "The contract amount is...", "document id": "contract-001", "page number": 14, "section": "Contract Amount", "chunk id": "contract-001-page14-chunk03" } This gives the retrieval system additional context and, more importantly, allows the generated result to point back to the original source. For reviewed business documents, this distinction is extremely important. A generated statement without evidence is difficult to trust. A generated statement with: Finding: The contract amount exceeds the configured threshold. Evidence: Contract.pdf — Page 14 is much easier for a human reviewer to validate. A common approach is to split documents by a fixed number of tokens. Every 500 tokens → one chunk This is simple, but document structure can be more important than chunk size. A better strategy can combine: Page 14 ├── Contract Overview │ ├── Contractor │ ├── Contract Number │ └── Contract Amount │ └── Payment Terms ├── Payment Schedule └── Conditions The resulting chunks carry both the content and its context. Pure vector search is powerful for semantic similarity. However, business documents often contain exact identifiers: ABC-2026-00125 ¥13,750,000 Project ID: PJ-10293 These values may be better handled by keyword or exact matching. This is where hybrid search becomes useful. Conceptually: User Query │ ├───────────────┐ ↓ ↓ Keyword Search Vector Search │ │ └───────┬───────┘ ↓ Result Ranking ↓ Top Evidence Keyword search can capture exact terms. Vector search can capture semantic meaning. Combining them gives the system more flexibility across different document types and query patterns. A common mistake is to evaluate a RAG system only by asking: “Does the LLM produce a good answer?” That is too late in the pipeline. We should separately evaluate retrieval. Precision How many retrieved documents are actually relevant? Recall How many of the relevant documents did we successfully retrieve? MRR Mean Reciprocal Rank How high does the first relevant result appear? These metrics help identify whether a problem comes from: Retrieval ↓ Context construction ↓ Prompt ↓ LLM generation Without separating these stages, it becomes difficult to determine why the system is failing. For business applications, free-form LLM responses can be difficult to validate. Instead of asking the model to return: I found that the amount appears to exceed... we can define a structured schema: { "finding": "Contract amount exceeds threshold", "severity": "warning", "value": 13750000, "threshold": 10000000, "evidence": { "document": "contract.pdf", "page": 14 } } Now the application can validate: before displaying the result to the user. This creates a much stronger boundary between the probabilistic LLM layer and the deterministic application layer. One of the most important design decisions is to treat evidence as part of the generated result—not as an optional UI feature. Finding │ ├── Generated statement │ ├── Supporting evidence │ ├── Source document │ └── Source page The application can then allow a reviewer to move directly from the finding to the relevant page. This creates a human verification loop: AI Finding ↓ Evidence ↓ Human Review ↓ Approve / Reject / Correct For systems used in document review, this workflow can be more valuable than simply trying to maximize the amount of text generated by the model. Another useful architectural principle is to distinguish between: What the model thinks is happening and What evidence supports that conclusion Finding Generation ↓ "Amount exceeds threshold" ↓ Evidence Retrieval ↓ Page 14 ↓ Source text / table ↓ Validation This allows the application to verify that an AI-generated finding actually has supporting evidence. It also makes debugging easier. If a finding is incorrect, we can ask: That is much more actionable than simply saying: “The AI hallucinated.” Putting these ideas together: ┌─────────────────┐ │ PDF Upload │ └────────┬────────┘ ↓ ┌───────────────────┐ │ OCR + Layout │ │ Understanding │ └─────────┬─────────┘ ↓ ┌───────────────────┐ │ Page-aware │ │ Document Model │ └─────────┬─────────┘ ↓ ┌───────────────────┐ │ Chunking + │ │ Metadata │ └─────────┬─────────┘ ↓ ┌───────────────────┐ │ Search Index │ │ Keyword + Vector │ └─────────┬─────────┘ ↓ User Query ↓ ┌───────────────────┐ │ Hybrid Retrieval │ └─────────┬─────────┘ ↓ ┌───────────────────┐ │ LLM Generation │ │ Structured Output │ └─────────┬─────────┘ ↓ ┌───────────────────┐ │ Evidence + │ │ Citation Validation│ └─────────┬─────────┘ ↓ Human Review The LLM is only one component of the system. The surrounding engineering determines whether the system is reliable enough for production. Building AI systems around real-world documents has changed how I think about RAG. The difficult part is rarely: “How do I call an LLM?” The difficult part is designing the entire pipeline around it. A production system needs to consider: Most importantly, the system should make it easy to verify what the AI is saying. For many enterprise AI applications, trust does not come from the model alone. It comes from the combination of: Good retrieval + structured generation + verifiable evidence + human review. That is the foundation I would use when designing a production RAG system for document-heavy applications. RAG should not be viewed simply as: “Give documents to an LLM.” A better mental model is: “Build a reliable evidence pipeline around an LLM.” Once you think about RAG this way, many engineering decisions—from page-aware ingestion to hybrid search and citation validation—become much clearer.