Building Ask Leya: Lessons Learned from Local ONNX Embeddings, Prisma Migrations, and Express Async Handlers

A developer detailed the architecture of Ask Leya, a chatbot that answers from a business's own content and hands off to a human when needed. The project uses local ONNX embeddings to avoid per-query costs and keep data on the server, but faces challenges such as cold starts and a 90-second production hang from a macOS binary. Other hurdles include managing a shared database with unreleased modules via Prisma and handling Express 4's lack of async rejection forwarding, which caused /chat to hang for 90 seconds per request.

Why build another chat widget? Because every widget I tried either followed a rigid script or confidently made things up. Ask Leya answers from a business's own content and hands off to a human when it can't. Here are three core architectural hurdles I ran into and how I solved them: No translation step, no API call, no per-query embedding cost, and nothing leaves the server to be embedded. The chat model then only has to phrase the answer back in Arabic, which is the easy half. The trade: A cold start while the model loads, and it pulls in sharp, which cost me a 90-second production hang when a macOS binary got rsynced onto a Linux box. Managing a database shared with unreleased modules The database is shared with the rest of my broader app under development — 67 tables total right now, many of which belong to unreleased modules. Since Prisma migrate tries to treat unreleased tables as drift, every migration currently requires hand-filtering additive statements via prisma migrate diff until the full suite ships. Express 4 doesn't forward async rejections An unguarded async handler that throws gives you no response at all — the caller just hangs until it times out. That took /chat down for 90 seconds a request before I tracked it down. The Stack Frontend: Next.js App Router, React with a dependency-free vanilla JS iframe widget so it can't collide with customer CSS . Backend: Node.js, Express. Database: Postgres Neon + pgvector. Real-time & AI: Socket.IO for live human handoff, gpt-4o-mini via OpenRouter for replies. Free tier, no credit card required. Features like WhatsApp, Instagram, and Messenger integrations are built out but kept behind the scenes until fully polished for self-serve.