How I Built a Local AI Orchestrator and City AI: My Journey as a Developer

Developer Raj Patil built Local AI Orchestrator, an offline, privacy-first AI system that runs on consumer hardware like an NVIDIA RTX 3050, achieving sub-100ms token generation times through LLM quantization. He also developed City AI, a platform that automates municipal feedback loops by categorizing and routing citizen complaints to the correct government departments.

Hi, I'm Raj Patil known online as Dream / lostxmusafir , an AI Engineer and Full-Stack Developer. In this article, I want to break down how I developed two of my most impactful projects: Local AI Orchestrator and City AI . You can find my full interactive portfolio here: Raj Patil AI Portfolio https://rajpatil-port.vercel.app/ . With growing data privacy concerns, relying on cloud-based LLM APIs isn't always feasible for enterprise applications. I built the Local AI Orchestrator to solve this. It's a completely offline, privacy-first AI system that runs locally on consumer hardware like my NVIDIA RTX 3050. Optimizing LLM quantization GGUF formats was critical to achieving sub-100ms token generation times on a local laptop GPU. It proved that robust, responsive AI applications don't always need expensive cloud servers. City AI is a platform built to automate municipal feedback loops. When citizens submit complaints like street light outages or road damage , the system categorizes, prioritizes, and routes them to the correct local government departments automatically. I believe the future of software lies at the intersection of high-performance web applications and intelligent AI orchestrations. Whether it's building a full-stack food delivery system like my Swiggy Clone or engineering cross-platform apps using Flutter like my mobile civic client Place.ai , my objective is to make software feel alive, smart, and exceptionally fast. Feel free to check out my open-source code and reach out if you'd like to collaborate on building next-generation AI platforms