{"slug": "how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists", "title": "How I Built a Production RAG System for Warehouse Operators — Not Data Scientists", "summary": "A solo engineer built Logistics Oracle, an AI-powered querying and forecasting tool for warehouse operators in UAE free zones, using a hybrid retrieval-augmented generation (RAG) system. The system combines PostgreSQL full-text search and vector similarity to handle real logistics data, and it runs on FastAPI, Redis, Groq, and Next.js, with no LangChain. The engineer reported a 45x speedup from Redis caching and emphasized that hybrid retrieval outperforms pure vector search for structured operational data.", "body_md": "A solo engineer's honest account of shipping an AI querying layer on top of real logistics data in the UAE — what worked, what broke, and what I'd do differently.\n\nMost RAG tutorials start with a PDF and a Jupyter notebook.\n\nReal warehouse operators don't have PDFs. They have spreadsheets, WhatsApp voice notes, half-filled ERP exports, and a shift manager who's been doing this for 14 years and keeps everything in his head.\n\nI spent the last several months building Logistics Oracle — an AI-powered querying and forecasting tool targeting SME operators in UAE free zones (JAFZA context). The goal wasn't to impress ML engineers. It was to let a warehouse supervisor ask \"which SKUs are we likely to run short on next week?\" in plain language and get a useful answer — not a Python traceback.\n\nHere's what I actually built, what the architecture looks like in production, and where the real complexity lives.\n\nBefore diving in, here's what's running in production:\n\n```\nLayer                 Technology\nAPI                FastAPI\nDatabase           PostgreSQL + Alembic migrations\nCache                  Redis\nLLM Inference          Groq (primary) / Ollama (local fallback)\nFrontend           Next.js\nAuth                   JWT + RBAC\nMonitoring         Prometheus\nDeploy                 Render (API) + Vercel (frontend)\n```\n\nNo LangChain. No bloated orchestration framework. Just deliberate plumbing.\n\nWhy RAG? (And Why Not Fine-Tuning?)\n\nThe question I get most: \"Why not just fine-tune a model on your logistics data?\"\n\nThree reasons:\n\nData freshness. Warehouse inventory changes daily. Fine-tuned models are snapshots. RAG retrieves from a live database — that's non-negotiable when a supervisor is asking about today's stock levels.\n\nAuditability. When an operator asks why the system flagged a SKU for reorder, they need to trace that back to actual records. RAG's retrieved context is inspectable. A fine-tuned model's weight updates are not.\n\nCost vs. iteration speed. Fine-tuning a meaningful logistics domain model requires volume and labeling budget. RAG on Groq with llama3-70b-8192 costs fractions of a cent per query and can be improved by changing a retrieval strategy in an afternoon.\n\nHere's how a natural-language query actually flows through the system:\n\n```\nUser Query\n    │\n    ▼\n[FastAPI endpoint]  ←── JWT auth middleware (RBAC check)\n    │\n    ▼\n[Redis cache lookup]  ──── HIT ──►  return cached response (~2ms)\n    │ MISS\n    ▼\n[Query Preprocessor]\n  - Intent classification (forecast / inventory / compliance / general)\n  - Entity extraction (SKU codes, dates, warehouse zones)\n    │\n    ▼\n[Retrieval Layer]\n  - PostgreSQL: structured inventory + transaction records\n  - Vector similarity: semantic search over product descriptions / notes\n    │\n    ▼\n[Context Assembly]\n  - Rank and truncate retrieved records to fit context window\n  - Inject domain-specific system prompt\n    │\n    ▼\n[Groq LLM — llama3-70b-8192]\n    │\n    ▼\n[Response Formatter]\n  - Strip hallucinated SKU codes (validation against DB)\n  - Format as operator-readable summary\n    │\n    ▼\nCache result in Redis (TTL: 5 min for inventory, 30 min for forecasts)\n    │\n    ▼\nReturn to Next.js frontend\n```\n\nThe Redis caching layer alone gave us ~45x speedup on repeated or near-identical queries. For a shift supervisor hitting the same forecast query every morning, this matters.\n\nMost RAG papers assume you have a large, clean corpus. SME logistics data is neither.\n\nWhat I found: hybrid retrieval outperforms pure vector search for structured operational data. PostgreSQL full-text search (tsvector) handles SKU code lookups and exact matches; vector similarity handles vague operator language like \"the stuff we got from that supplier last quarter.\"\n\nThe retrieval pipeline runs both, scores results with a weighted merge, and passes the top-k to the LLM. Tuning the weight ratio between the two (currently 0.6 structured / 0.4 semantic) took more iteration than any other part of the system.\n\nLLMs confidently fabricate product codes. In a logistics context, a hallucinated SKU isn't an inconvenience — it's a picking error or a bad purchase order.\n\nThe fix: every quantity, SKU code, or date in the LLM's response is validated against the database before it reaches the user. If the model references a SKU that doesn't exist in the current inventory snapshot, that claim is flagged and either corrected or stripped.\n\nThis added a round-trip but it's non-negotiable. Operators lost trust in the system the moment they caught a single bad SKU. They never forget.\n\n*The hardest problem wasn't technical.*\n\nA warehouse supervisor asking \"are we good on the floor this week?\" doesn't mean \"retrieve inventory records and generate a status summary.\" It means: check the three SKUs we always run short on, flag anything below reorder threshold, and also check if there's a delivery due that would resolve it.\n\nBridging that intent gap required building a lightweight query intent classifier that maps natural language patterns to structured retrieval strategies before the LLM ever sees the query. This runs fast (small local model via Ollama) and routes the query to the right combination of retrieval paths.\n\nThe system uses JWT with role-based access control across three roles:\n\noperator — can query inventory and forecasts for their assigned warehouse zone\n\nmanager — full read + can trigger manual reorder alerts\n\nadmin — full access including raw data exports\n\nThis matters because in a free zone context, different clients (tenants) share infrastructure but must never see each other's data. Every database query is scoped to tenant_id before it hits retrieval. The LLM prompt includes the tenant context but never receives cross-tenant records.\n\n36 automated tests cover the critical paths. But tests don't tell you what's slow in production.\n\nPrometheus scrapes metrics at every stage of the pipeline:\n\nQuery latency by intent type\n\nCache hit rate\n\nRetrieval precision (proxied via user follow-up rate — if operators rephrase immediately, retrieval probably failed them)\n\nLLM token usage per query type\n\nThe metric that surprised me most: forecast queries take 3x longer than inventory lookups not because of the LLM — but because of the aggregation SQL before retrieval. That's where the optimization effort actually belongs.\n\nUse structured output from day one. Having the LLM return a free-text response that I then parse is fragile. Groq supports JSON mode — I should have standardized on structured outputs from the first sprint. Retrofitting it halfway through cost me a week.\n\nBuild the operator feedback loop earlier. I shipped a technically solid system before I had a real user. The intent classifier I was proud of missed entire categories of how actual supervisors phrase questions. You cannot design around this from the outside.\n\nDon't defer monitoring. I added Prometheus in Phase 1 polish, not from the start. Flying blind on latency for the first two months meant I optimized the wrong things.\n\nAfter Phase 1, the system scores around 8.8/10 across the technical criteria I set — latency, accuracy, auth correctness, test coverage.\n\nBut the benchmark that matters is simpler: will a warehouse supervisor in JAFZA choose this over the spreadsheet they've used for 6 years?\n\nThat's the question I'm taking to real users now. Everything else is just engineering.\n\nThe next layer I'm exploring is a LangGraph agent that can handle multi-step reasoning — for example: \"Flag everything below reorder threshold, check if there's an open PO that covers it, and only alert me on the gaps.\" That's a workflow, not a single query, and it needs agent-style orchestration.\n\nBut that's Phase 2. Phase 1's lesson was: ship something real operators can break before you build the next clever thing.\n\nIf you're building AI tools for operational SMEs — especially in logistics or supply chain — I'd genuinely like to compare notes. The gap between \"works in demo\" and \"trusted in a warehouse\" is where most of the interesting problems live.\n\nFind me on LinkedIn or drop a comment below.", "url": "https://wpnews.pro/news/how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists", "canonical_source": "https://dev.to/dulasi_nethma_e5461557016/how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists-1dn3", "published_at": "2026-09-03 09:56:21+00:00", "updated_at": "2026-09-03 10:24:09.317127+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-infrastructure", "developer-tools"], "entities": ["Logistics Oracle", "Groq", "Ollama", "FastAPI", "PostgreSQL", "Redis", "Next.js", "JAFZA"], "alternates": {"html": "https://wpnews.pro/news/how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists", "markdown": "https://wpnews.pro/news/how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists.md", "text": "https://wpnews.pro/news/how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists.txt", "jsonld": "https://wpnews.pro/news/how-i-built-a-production-rag-system-for-warehouse-operators-not-data-scientists.jsonld"}}