Building an AI Document Intelligence System: Architecture, LangChain, and Production Lessons

A developer built AI-DocumentIntelligence, an open-source, self-hostable, provider-agnostic RAG platform for document upload, processing, and natural-language Q&A. The system uses LangChain for orchestration, PostgreSQL with pgvector as the vector store, and a React/Node.js stack, with LLM providers OpenAI or Anthropic Claude selectable via a configuration flag rather than hardcoded. The developer said the goal was to build something they'd be comfortable recommending to any delivery team handling sensitive documents: transparent, swappable, and auditable.

How I built a provider-agnostic RAG platform for document Q&A - and what years of delivering document-heavy digital services taught me about doing it properly. Every organisation I've worked with - public sector, regulated enterprise, or otherwise - has the same quiet bottleneck: information locked inside PDFs, DOCX files, and scanned reports that nobody has time to read in full. A caseworker needs one clause from a 40-page policy document. An analyst needs a number buried on page 22 of a compliance report. The document exists, technically "available," but functionally unsearchable. Having spent several years delivering large, document-heavy digital services, I saw this pattern constantly, across departments and industries: teams either paid for a closed-source document AI product with a vague pricing model and a data-residency question mark, or they lived with the manual search. So I built AI-DocumentIntelligence https://github.com/Srameshgitnow/AI-DocumentIntelligence - an open-source, self-hostable, provider-agnostic RAG Retrieval-Augmented Generation platform for document upload, processing, and natural-language Q&A. The goal wasn't to build another wrapper around an API. It was to build something I'd be comfortable recommending to any delivery team handling sensitive documents: transparent, swappable, and auditable. At its core, the app does four things: pgvector That last point matters more than it sounds. Most tutorial-grade RAG projects hardcode a single LLM provider. In a real procurement context - any regulated department or enterprise - that's a non-starter - you need to be able to swap providers for cost, compliance, or availability reasons without a rewrite. So LLM PROVIDER is a config flag, not an architectural decision baked into the code. | Layer | Technology | |---|---| | Frontend | React 18 + TypeScript | | Backend | Node.js + Express + TypeScript | | AI / orchestration | LangChain | | Vector store | PostgreSQL + pgvector | | LLMs | OpenAI or Anthropic Claude configurable | | Dev tooling | Docker, ts-node, dotenv | I deliberately kept this to a stack that's boring in the best sense - nothing here requires a team to learn a new deployment paradigm. If your organisation can already run a Node.js service and a PostgreSQL database, you can run this. ┌────────────┐ ┌──────────────────┐ ┌────────────────────┐ │ React UI │ ───► │ Express API │ ───► │ Document Processor │ │ upload+chat │ │ TypeScript │ │ chunking / split │ └────────────┘ └──────────────────┘ └────────────────────┘ │ │ ▼ ▼ ┌──────────────────┐ ┌────────────────────┐ │ LangChain Layer │ ◄──► │ PostgreSQL + pgvector│ │ OpenAI / Claude │ │ embeddings store │ └──────────────────┘ └────────────────────┘ The chunking and embedding pipeline runs once per upload. Query time is a straightforward retrieve-then-generate flow: embed the question, do a similarity search against pgvector , pass the top-k chunks to the selected LLM with a grounding prompt, and return an answer plus the chat history for that document. Nothing exotic. That's intentional - the value isn't in a clever novel architecture, it's in getting the boring parts provider abstraction, chunk quality, error handling around missing credentials actually right. Switching LLM providers is a config read, not a code branch buried three files deep: js // backend/src/llm/provider.ts simplified const provider = process.env.LLM PROVIDER; // "openai" | "anthropic" export function getChatModel { if provider === "anthropic" { return new ChatAnthropic { apiKey: process.env.ANTHROPIC API KEY, model: process.env.ANTHROPIC MODEL, } ; } return new ChatOpenAI { apiKey: process.env.OPENAI API KEY, } ; } Chunking uses LangChain's recursive splitter, tuned for document Q&A rather than raw ingestion: js const splitter = new RecursiveCharacterTextSplitter { chunkSize: 1000, chunkOverlap: 150, } ; const chunks = await splitter.splitDocuments rawDocs ; The overlap matters more than most tutorials admit - too little, and you clip an answer in half across a chunk boundary; too much, and you're paying to embed and search the same sentence three times. 150 on a 1000 -character chunk was the balance that held up best across the mixed PDF/DOCX test set I used. Retrieval and generation stay separated , so either half is independently testable - I can validate that retrieval returns the right chunks without spending API credits on generation, which matters a lot when you're iterating on an unpaid OpenAI tier. What went right: What was genuinely hard: docker compose up " .env.example , explicit DB NAME expectations, a documented port-conflict fallback - took longer than the RAG logic itself, and it's the part that actually determines whether anyone else can use the project. I'm building this alongside a small portfolio of open-source AI engineering projects - a policy Q&A RAG system for citizen-facing services, an agentic PR-review bot, a voice-to-Agile-user-story generator - all reflecting the same thread: applying LLM orchestration LangChain/LangGraph to real workflow problems I've encountered across large-scale digital delivery, not toy demos. If that combination - production delivery experience plus applied open-source AI engineering - is useful context for anyone evaluating this work, that's very much the point of publishing it. The repo is open source and set up for local development in a few commands: git clone https://github.com/Srameshgitnow/AI-DocumentIntelligence.git cd AI-DocumentIntelligence docker compose up --build Full setup instructions, environment variables, and troubleshooting are in the README https://github.com/Srameshgitnow/AI-DocumentIntelligence . If you try it, hit a bug, or have thoughts on the provider-abstraction approach, I'd genuinely like to hear them - issues and PRs are open. And if the project is useful to you, a ⭐ on the repo goes a long way toward other people finding it: 👉 github.com/Srameshgitnow/AI-DocumentIntelligence https://github.com/Srameshgitnow/AI-DocumentIntelligence I'm a full-stack / AI engineer React, Node.js, LangChain/LangGraph with a background delivering large-scale digital services. I write about applied AI engineering and open-source tooling - follow along for the next post in this series.