{"slug": "building-an-ai-document-intelligence-system-architecture-langchain-and-lessons", "title": "Building an AI Document Intelligence System: Architecture, LangChain, and Production Lessons", "summary": "A developer built AI-DocumentIntelligence, an open-source, self-hostable, provider-agnostic RAG platform for document upload, processing, and natural-language Q&A. The system uses LangChain for orchestration, PostgreSQL with pgvector as the vector store, and a React/Node.js stack, with LLM providers OpenAI or Anthropic Claude selectable via a configuration flag rather than hardcoded. The developer said the goal was to build something they'd be comfortable recommending to any delivery team handling sensitive documents: transparent, swappable, and auditable.", "body_md": "*How I built a provider-agnostic RAG platform for document Q&A - and what years of delivering document-heavy digital services taught me about doing it properly.*\n\nEvery organisation I've worked with - public sector, regulated enterprise, or otherwise - has the same quiet bottleneck: information locked inside PDFs, DOCX files, and scanned reports that nobody has time to read in full. A caseworker needs one clause from a 40-page policy document. An analyst needs a number buried on page 22 of a compliance report. The document exists, technically \"available,\" but functionally unsearchable.\n\nHaving spent several years delivering large, document-heavy digital services, I saw this pattern constantly, across departments and industries: teams either paid for a closed-source document AI product with a vague pricing model and a data-residency question mark, or they lived with the manual search.\n\nSo I built [**AI-DocumentIntelligence**](https://github.com/Srameshgitnow/AI-DocumentIntelligence) - an open-source, self-hostable, provider-agnostic RAG (Retrieval-Augmented Generation) platform for document upload, processing, and natural-language Q&A. The goal wasn't to build another wrapper around an API. It was to build something I'd be comfortable recommending to any delivery team handling sensitive documents: transparent, swappable, and auditable.\n\nAt its core, the app does four things:\n\n`pgvector`\nThat last point matters more than it sounds. Most tutorial-grade RAG projects hardcode a single LLM provider. In a real procurement context - any regulated department or enterprise - that's a non-starter - you need to be able to swap providers for cost, compliance, or availability reasons without a rewrite. So `LLM_PROVIDER` is a config flag, not an architectural decision baked into the code.\n\n| Layer | Technology | \n|---|---|\n| Frontend | React 18 + TypeScript | \n| Backend | Node.js + Express + TypeScript | \n| AI / orchestration | LangChain | \n| Vector store | PostgreSQL + pgvector | \n| LLMs | OpenAI **or** Anthropic Claude (configurable) | \n| Dev tooling | Docker, ts-node, dotenv | \n\nI deliberately kept this to a stack that's boring in the best sense - nothing here requires a team to learn a new deployment paradigm. If your organisation can already run a Node.js service and a PostgreSQL database, you can run this.\n\n```\n ┌────────────┐      ┌──────────────────┐      ┌────────────────────┐\n │  React UI   │ ───► │  Express API      │ ───► │  Document Processor │\n │ (upload+chat)│      │ (TypeScript)       │      │ (chunking / split)   │\n └────────────┘      └──────────────────┘      └────────────────────┘\n                                │                          │\n                                ▼                          ▼\n                      ┌──────────────────┐      ┌────────────────────┐\n                      │  LangChain Layer   │ ◄──► │ PostgreSQL + pgvector│\n                      │ (OpenAI / Claude)    │      │  (embeddings store)   │\n                      └──────────────────┘      └────────────────────┘\n```\n\nThe chunking and embedding pipeline runs once per upload. Query time is a straightforward retrieve-then-generate flow: embed the question, do a similarity search against `pgvector`, pass the top-k chunks to the selected LLM with a grounding prompt, and return an answer plus the chat history for that document.\n\nNothing exotic. That's intentional - the value isn't in a clever novel architecture, it's in getting the boring parts (provider abstraction, chunk quality, error handling around missing credentials) actually right.\n\n**Switching LLM providers is a config read, not a code branch buried three files deep:**\n\n``` js\n// backend/src/llm/provider.ts (simplified)\nconst provider = process.env.LLM_PROVIDER; // \"openai\" | \"anthropic\"\n\nexport function getChatModel() {\n  if (provider === \"anthropic\") {\n    return new ChatAnthropic({\n      apiKey: process.env.ANTHROPIC_API_KEY,\n      model: process.env.ANTHROPIC_MODEL,\n    });\n  }\n  return new ChatOpenAI({\n    apiKey: process.env.OPENAI_API_KEY,\n  });\n}\n```\n\n**Chunking uses LangChain's recursive splitter, tuned for document Q&A rather than raw ingestion:**\n\n``` js\nconst splitter = new RecursiveCharacterTextSplitter({\n  chunkSize: 1000,\n  chunkOverlap: 150,\n});\nconst chunks = await splitter.splitDocuments(rawDocs);\n```\n\nThe overlap matters more than most tutorials admit - too little, and you clip an answer in half across a chunk boundary; too much, and you're paying to embed and search the same sentence three times. `150` on a `1000`-character chunk was the balance that held up best across the mixed PDF/DOCX test set I used.\n\n**Retrieval and generation stay separated**, so either half is independently testable - I can validate that retrieval returns the right chunks without spending API credits on generation, which matters a lot when you're iterating on an unpaid OpenAI tier.\n\n**What went right:**\n\n**What was genuinely hard:**\n\n`docker compose up`\"`.env.example`, explicit `DB_NAME` expectations, a documented port-conflict fallback - took longer than the RAG logic itself, and it's the part that actually determines whether anyone else can use the project.\nI'm building this alongside a small portfolio of open-source AI engineering projects - a policy Q&A RAG system for citizen-facing services, an agentic PR-review bot, a voice-to-Agile-user-story generator - all reflecting the same thread: applying LLM orchestration (LangChain/LangGraph) to real workflow problems I've encountered across large-scale digital delivery, not toy demos. If that combination - production delivery experience plus applied open-source AI engineering - is useful context for anyone evaluating this work, that's very much the point of publishing it.\n\nThe repo is open source and set up for local development in a few commands:\n\n```\ngit clone https://github.com/Srameshgitnow/AI-DocumentIntelligence.git\ncd AI-DocumentIntelligence\ndocker compose up --build\n```\n\nFull setup instructions, environment variables, and troubleshooting are in the [README](https://github.com/Srameshgitnow/AI-DocumentIntelligence).\n\nIf you try it, hit a bug, or have thoughts on the provider-abstraction approach, I'd genuinely like to hear them - issues and PRs are open. And if the project is useful to you, a ⭐ on the repo goes a long way toward other people finding it:\n\n👉 [github.com/Srameshgitnow/AI-DocumentIntelligence](https://github.com/Srameshgitnow/AI-DocumentIntelligence)\n\n*I'm a full-stack / AI engineer (React, Node.js, LangChain/LangGraph) with a background delivering large-scale digital services. I write about applied AI engineering and open-source tooling - follow along for the next post in this series.*", "url": "https://wpnews.pro/news/building-an-ai-document-intelligence-system-architecture-langchain-and-lessons", "canonical_source": "https://dev.to/ramesh_s_a8f0867d239e927c/building-an-ai-document-intelligence-system-architecture-langchain-and-production-lessons-563f", "published_at": "2026-09-14 16:29:01+00:00", "updated_at": "2026-09-14 16:55:29.177310+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-tools", "ai-infrastructure"], "entities": ["AI-DocumentIntelligence", "LangChain", "PostgreSQL", "pgvector", "OpenAI", "Anthropic", "Claude", "React"], "alternates": {"html": "https://wpnews.pro/news/building-an-ai-document-intelligence-system-architecture-langchain-and-lessons", "markdown": "https://wpnews.pro/news/building-an-ai-document-intelligence-system-architecture-langchain-and-lessons.md", "text": "https://wpnews.pro/news/building-an-ai-document-intelligence-system-architecture-langchain-and-lessons.txt", "jsonld": "https://wpnews.pro/news/building-an-ai-document-intelligence-system-architecture-langchain-and-lessons.jsonld"}}