# Building an AI Document Intelligence System: Architecture, LangChain, and Production Lessons

> Source: <https://dev.to/ramesh_s_a8f0867d239e927c/building-an-ai-document-intelligence-system-architecture-langchain-and-production-lessons-563f>
> Published: 2026-09-14 16:29:01+00:00

*How I built a provider-agnostic RAG platform for document Q&A - and what years of delivering document-heavy digital services taught me about doing it properly.*

Every organisation I've worked with - public sector, regulated enterprise, or otherwise - has the same quiet bottleneck: information locked inside PDFs, DOCX files, and scanned reports that nobody has time to read in full. A caseworker needs one clause from a 40-page policy document. An analyst needs a number buried on page 22 of a compliance report. The document exists, technically "available," but functionally unsearchable.

Having spent several years delivering large, document-heavy digital services, I saw this pattern constantly, across departments and industries: teams either paid for a closed-source document AI product with a vague pricing model and a data-residency question mark, or they lived with the manual search.

So I built [**AI-DocumentIntelligence**](https://github.com/Srameshgitnow/AI-DocumentIntelligence) - an open-source, self-hostable, provider-agnostic RAG (Retrieval-Augmented Generation) platform for document upload, processing, and natural-language Q&A. The goal wasn't to build another wrapper around an API. It was to build something I'd be comfortable recommending to any delivery team handling sensitive documents: transparent, swappable, and auditable.

At its core, the app does four things:

`pgvector`
That last point matters more than it sounds. Most tutorial-grade RAG projects hardcode a single LLM provider. In a real procurement context - any regulated department or enterprise - that's a non-starter - you need to be able to swap providers for cost, compliance, or availability reasons without a rewrite. So `LLM_PROVIDER` is a config flag, not an architectural decision baked into the code.

| Layer | Technology | 
|---|---|
| Frontend | React 18 + TypeScript | 
| Backend | Node.js + Express + TypeScript | 
| AI / orchestration | LangChain | 
| Vector store | PostgreSQL + pgvector | 
| LLMs | OpenAI **or** Anthropic Claude (configurable) | 
| Dev tooling | Docker, ts-node, dotenv | 

I deliberately kept this to a stack that's boring in the best sense - nothing here requires a team to learn a new deployment paradigm. If your organisation can already run a Node.js service and a PostgreSQL database, you can run this.

```
 ┌────────────┐      ┌──────────────────┐      ┌────────────────────┐
 │  React UI   │ ───► │  Express API      │ ───► │  Document Processor │
 │ (upload+chat)│      │ (TypeScript)       │      │ (chunking / split)   │
 └────────────┘      └──────────────────┘      └────────────────────┘
                                │                          │
                                ▼                          ▼
                      ┌──────────────────┐      ┌────────────────────┐
                      │  LangChain Layer   │ ◄──► │ PostgreSQL + pgvector│
                      │ (OpenAI / Claude)    │      │  (embeddings store)   │
                      └──────────────────┘      └────────────────────┘
```

The chunking and embedding pipeline runs once per upload. Query time is a straightforward retrieve-then-generate flow: embed the question, do a similarity search against `pgvector`, pass the top-k chunks to the selected LLM with a grounding prompt, and return an answer plus the chat history for that document.

Nothing exotic. That's intentional - the value isn't in a clever novel architecture, it's in getting the boring parts (provider abstraction, chunk quality, error handling around missing credentials) actually right.

**Switching LLM providers is a config read, not a code branch buried three files deep:**

``` js
// backend/src/llm/provider.ts (simplified)
const provider = process.env.LLM_PROVIDER; // "openai" | "anthropic"

export function getChatModel() {
  if (provider === "anthropic") {
    return new ChatAnthropic({
      apiKey: process.env.ANTHROPIC_API_KEY,
      model: process.env.ANTHROPIC_MODEL,
    });
  }
  return new ChatOpenAI({
    apiKey: process.env.OPENAI_API_KEY,
  });
}
```

**Chunking uses LangChain's recursive splitter, tuned for document Q&A rather than raw ingestion:**

``` js
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 150,
});
const chunks = await splitter.splitDocuments(rawDocs);
```

The overlap matters more than most tutorials admit - too little, and you clip an answer in half across a chunk boundary; too much, and you're paying to embed and search the same sentence three times. `150` on a `1000`-character chunk was the balance that held up best across the mixed PDF/DOCX test set I used.

**Retrieval and generation stay separated**, so either half is independently testable - I can validate that retrieval returns the right chunks without spending API credits on generation, which matters a lot when you're iterating on an unpaid OpenAI tier.

**What went right:**

**What was genuinely hard:**

`docker compose up`"`.env.example`, explicit `DB_NAME` expectations, a documented port-conflict fallback - took longer than the RAG logic itself, and it's the part that actually determines whether anyone else can use the project.
I'm building this alongside a small portfolio of open-source AI engineering projects - a policy Q&A RAG system for citizen-facing services, an agentic PR-review bot, a voice-to-Agile-user-story generator - all reflecting the same thread: applying LLM orchestration (LangChain/LangGraph) to real workflow problems I've encountered across large-scale digital delivery, not toy demos. If that combination - production delivery experience plus applied open-source AI engineering - is useful context for anyone evaluating this work, that's very much the point of publishing it.

The repo is open source and set up for local development in a few commands:

```
git clone https://github.com/Srameshgitnow/AI-DocumentIntelligence.git
cd AI-DocumentIntelligence
docker compose up --build
```

Full setup instructions, environment variables, and troubleshooting are in the [README](https://github.com/Srameshgitnow/AI-DocumentIntelligence).

If you try it, hit a bug, or have thoughts on the provider-abstraction approach, I'd genuinely like to hear them - issues and PRs are open. And if the project is useful to you, a ⭐ on the repo goes a long way toward other people finding it:

👉 [github.com/Srameshgitnow/AI-DocumentIntelligence](https://github.com/Srameshgitnow/AI-DocumentIntelligence)

*I'm a full-stack / AI engineer (React, Node.js, LangChain/LangGraph) with a background delivering large-scale digital services. I write about applied AI engineering and open-source tooling - follow along for the next post in this series.*