A solo engineer's honest account of shipping an AI querying layer on top of real logistics data in the UAE — what worked, what broke, and what I'd do differently.
Most RAG tutorials start with a PDF and a Jupyter notebook.
Real warehouse operators don't have PDFs. They have spreadsheets, WhatsApp voice notes, half-filled ERP exports, and a shift manager who's been doing this for 14 years and keeps everything in his head.
I spent the last several months building Logistics Oracle — an AI-powered querying and forecasting tool targeting SME operators in UAE free zones (JAFZA context). The goal wasn't to impress ML engineers. It was to let a warehouse supervisor ask "which SKUs are we likely to run short on next week?" in plain language and get a useful answer — not a Python traceback.
Here's what I actually built, what the architecture looks like in production, and where the real complexity lives.
Before diving in, here's what's running in production:
Layer Technology
API FastAPI
Database PostgreSQL + Alembic migrations
Cache Redis
LLM Inference Groq (primary) / Ollama (local fallback)
Frontend Next.js
Auth JWT + RBAC
Monitoring Prometheus
Deploy Render (API) + Vercel (frontend)
No LangChain. No bloated orchestration framework. Just deliberate plumbing.
Why RAG? (And Why Not Fine-Tuning?)
The question I get most: "Why not just fine-tune a model on your logistics data?"
Three reasons:
Data freshness. Warehouse inventory changes daily. Fine-tuned models are snapshots. RAG retrieves from a live database — that's non-negotiable when a supervisor is asking about today's stock levels.
Auditability. When an operator asks why the system flagged a SKU for reorder, they need to trace that back to actual records. RAG's retrieved context is inspectable. A fine-tuned model's weight updates are not.
Cost vs. iteration speed. Fine-tuning a meaningful logistics domain model requires volume and labeling budget. RAG on Groq with llama3-70b-8192 costs fractions of a cent per query and can be improved by changing a retrieval strategy in an afternoon.
Here's how a natural-language query actually flows through the system:
User Query
│
▼
[FastAPI endpoint] ←── JWT auth middleware (RBAC check)
│
▼
[Redis cache lookup] ──── HIT ──► return cached response (~2ms)
│ MISS
▼
[Query Preprocessor]
- Intent classification (forecast / inventory / compliance / general)
- Entity extraction (SKU codes, dates, warehouse zones)
│
▼
[Retrieval Layer]
- PostgreSQL: structured inventory + transaction records
- Vector similarity: semantic search over product descriptions / notes
│
▼
[Context Assembly]
- Rank and truncate retrieved records to fit context window
- Inject domain-specific system prompt
│
▼
[Groq LLM — llama3-70b-8192]
│
▼
[Response Formatter]
- Strip hallucinated SKU codes (validation against DB)
- Format as operator-readable summary
│
▼
Cache result in Redis (TTL: 5 min for inventory, 30 min for forecasts)
│
▼
Return to Next.js frontend
The Redis caching layer alone gave us ~45x speedup on repeated or near-identical queries. For a shift supervisor hitting the same forecast query every morning, this matters.
Most RAG papers assume you have a large, clean corpus. SME logistics data is neither.
What I found: hybrid retrieval outperforms pure vector search for structured operational data. PostgreSQL full-text search (tsvector) handles SKU code lookups and exact matches; vector similarity handles vague operator language like "the stuff we got from that supplier last quarter."
The retrieval pipeline runs both, scores results with a weighted merge, and passes the top-k to the LLM. Tuning the weight ratio between the two (currently 0.6 structured / 0.4 semantic) took more iteration than any other part of the system.
LLMs confidently fabricate product codes. In a logistics context, a hallucinated SKU isn't an inconvenience — it's a picking error or a bad purchase order.
The fix: every quantity, SKU code, or date in the LLM's response is validated against the database before it reaches the user. If the model references a SKU that doesn't exist in the current inventory snapshot, that claim is flagged and either corrected or stripped.
This added a round-trip but it's non-negotiable. Operators lost trust in the system the moment they caught a single bad SKU. They never forget.
The hardest problem wasn't technical.
A warehouse supervisor asking "are we good on the floor this week?" doesn't mean "retrieve inventory records and generate a status summary." It means: check the three SKUs we always run short on, flag anything below reorder threshold, and also check if there's a delivery due that would resolve it.
Bridging that intent gap required building a lightweight query intent classifier that maps natural language patterns to structured retrieval strategies before the LLM ever sees the query. This runs fast (small local model via Ollama) and routes the query to the right combination of retrieval paths.
The system uses JWT with role-based access control across three roles:
operator — can query inventory and forecasts for their assigned warehouse zone
manager — full read + can trigger manual reorder alerts
admin — full access including raw data exports
This matters because in a free zone context, different clients (tenants) share infrastructure but must never see each other's data. Every database query is scoped to tenant_id before it hits retrieval. The LLM prompt includes the tenant context but never receives cross-tenant records.
36 automated tests cover the critical paths. But tests don't tell you what's slow in production.
Prometheus scrapes metrics at every stage of the pipeline:
Query latency by intent type
Cache hit rate
Retrieval precision (proxied via user follow-up rate — if operators rephrase immediately, retrieval probably failed them)
LLM token usage per query type
The metric that surprised me most: forecast queries take 3x longer than inventory lookups not because of the LLM — but because of the aggregation SQL before retrieval. That's where the optimization effort actually belongs.
Use structured output from day one. Having the LLM return a free-text response that I then parse is fragile. Groq supports JSON mode — I should have standardized on structured outputs from the first sprint. Retrofitting it halfway through cost me a week.
Build the operator feedback loop earlier. I shipped a technically solid system before I had a real user. The intent classifier I was proud of missed entire categories of how actual supervisors phrase questions. You cannot design around this from the outside.
Don't defer monitoring. I added Prometheus in Phase 1 polish, not from the start. Flying blind on latency for the first two months meant I optimized the wrong things.
After Phase 1, the system scores around 8.8/10 across the technical criteria I set — latency, accuracy, auth correctness, test coverage.
But the benchmark that matters is simpler: will a warehouse supervisor in JAFZA choose this over the spreadsheet they've used for 6 years?
That's the question I'm taking to real users now. Everything else is just engineering.
The next layer I'm exploring is a LangGraph agent that can handle multi-step reasoning — for example: "Flag everything below reorder threshold, check if there's an open PO that covers it, and only alert me on the gaps." That's a workflow, not a single query, and it needs agent-style orchestration.
But that's Phase 2. Phase 1's lesson was: ship something real operators can break before you build the next clever thing.
If you're building AI tools for operational SMEs — especially in logistics or supply chain — I'd genuinely like to compare notes. The gap between "works in demo" and "trusted in a warehouse" is where most of the interesting problems live.
Find me on LinkedIn or drop a comment below.