What Actually Happens When You Build a Production RAG Chatbot on Amazon Bedrock A developer built a production RAG document Q&A chatbot on Amazon Bedrock using NestJS, PostgreSQL with pgvector, hybrid search with reciprocal rank fusion, prompt caching, and Claude, and documented the architecture and the failure modes it addresses. The writeup argues that most RAG degradation is silent — stale documents, mismatched error codes, and ineffective caching — and that retrieval architecture, not prompt engineering, determines success. The system splits ingestion and query paths, using Cohere embeddings with distinct input types for documents and queries and a two-level chunker with header injection. How we built a document Q&A system with NestJS, PostgreSQL, pgvector, Hybrid Search RRF , Prompt Caching, and Claude on Amazon Bedrock. Building a demo chatbot is easy. Building one that consistently answers correctly after weeks of document updates, changing requirements, and real user traffic is where things become interesting. Most Retrieval-Augmented Generation RAG failures don't look like failures. The API returns 200. The model responds. Infrastructure dashboards stay green. Yet answer quality gradually gets worse. You ask about error code E-4021 and receive documentation for E-4012 . A recently updated document never appears in search results. Prompt caching is enabled, but costs don't go down. Hybrid search is implemented, but retrieval quality barely improves. Nothing crashes. Nothing raises an alert. The chatbot simply becomes less useful over time. For teams involved in application software development https://relipa.global/app-development-services-company/ , the real work begins after the first successful response. Retrieval quality, chunking strategy, embedding consistency, prompt caching, document lifecycle management, and hallucination control quickly become more important than model selection itself. This article walks through a complete production-oriented RAG implementation built with: More importantly, it explains why each component exists, what problem it solves, and which production issues it prevents. RAG Retrieval-Augmented Generation does not ask the LLM directly. Before each question, the system: The consequence of this design: The problem "how does the LLM know my data" becomes a search problem. If retrieval is weak, the chatbot is weak, because no prompt can produce information that isn't in the context. This distinction is easy to underestimate. Many teams spend weeks experimenting with prompts while retrieval quality remains unchanged. In reality, most enterprise application software development projects that incorporate AI succeed or fail based on retrieval architecture rather than prompt engineering. That is why most of this article is about chunking documents and search , not about prompts. Compared with the alternatives: A RAG system has three stages: The three stages of RAG: documents are indexed ahead of time, then each question goes through retrieval and then generation ══ OFFLINE ══════════════════════════════════════════════════════════ data/in/ .md|txt|json │ ▼ LoaderService ──► ChunkerService ──► Bedrock Cohere Embed ──► Postgres read files 2-level split + input type = chunks table header injection search document vector + tsvector ══ ONLINE ═══════════════════════════════════════════════════════════ POST /api/chat { question } │ ▼ SearchService.hybridSearch ├─ embedQuery Cohere, input type = search query ├─ DenseSearch HNSW index ─┐ run in parallel ├─ KeywordSearch GIN index ─┘ └─ reciprocalRankFusion ──► top 5 chunks │ ▼ PromptBuilder ── system + few-shot: CACHED + context + question: not cached │ ▼ Bedrock Claude, InvokeModel │ ▼ { answer, citations, usage } Folder layout, one folder per stage: src/ ├── config/ all config, read from env ├── database/ schema.sql + pg.Pool wrapper no ORM ├── llm/ LlmService interface + Bedrock implementation ├── ingest/ loader → chunker → repository → ingest.service ├── search/ dense · keyword · rrf · search.service ├── prompt/ system prompt · few-shot · prompt builder └── chat/ wires the three steps into one RAG loop + HTTP API scripts/ db-init · ingest · search CLI test/ unit tests for chunker, rrf, prompt builder This architecture may look surprisingly simple. That is intentional. Every stage has a single responsibility and can be tested independently. When retrieval quality drops, we can inspect retrieval. When generation quality drops, we can inspect prompts. When ingest breaks, we can inspect indexing. The separation becomes extremely valuable once the system grows beyond a proof of concept. The diagram in section 2 shows the blocks. This section slows down: it takes one concrete document and one concrete question and follows them through each function, each SQL query, and each Bedrock call. Once you have read this section, the code in the later steps is just detail. The sample data used throughout is data/in/warranty.md : Warranty Policy Warranty period All products are covered by a warranty of 12 months from the date of purchase. Returns Products with a manufacturing defect can be exchanged within the first 7 days. This flow runs offline via npm run ingest , not over HTTP. Re-run it whenever documents change. npm run ingest --