100+ Users, Free API Keys and ₹0 Cloud Budget: How I Built Kairos

A developer built Kairos, a retrieval-augmented generation (RAG) question-answering system for a departmental event, serving over 100 users on a ₹0 infrastructure budget. The system routes requests across 20 API lanes using free Gemini and OpenRouter/Nemotron tiers with PostgreSQL-based locking, and runs locally on a laptop exposed via ngrok, with Qdrant handling vector search and each team limited to 10 questions.

I wanted to build a simple RAG system. That sentence aged badly. The original problem was actually pretty simple. We were organizing a departmental event called Kairos , where participants had to study company information and create pitch decks. The organizers had a lot of company reports. The participants had to read them. And the obvious solution was: Print everything. Except there was one tiny problem. There were a lot of pages. And there were a lot of participants. Printing enough copies for everyone wasn't exactly friendly to the budget. So I had a stupidly simple thought: Why don't we just put all the documents into an AI system and let participants ask questions? And that's how I accidentally created a much bigger problem for myself. If you've built RAG systems before, you know how innocent that sentence sounds. Upload PDFs → chunk them → embed them → retrieve relevant chunks → send them to an LLM. Easy. Until you actually try to make people use it. I needed: And I had one major constraint: Not "small startup budget." I mean ₹0 infrastructure budget . The next question was obvious: Which LLM should I use? Paid APIs? No. I didn't have money for that. So I started looking at free API tiers. I ended up using Gemini as the primary model and OpenRouter/Nemotron as a fallback. Problem solved? Absolutely not. Free APIs have limits. And when you have one user asking a question, rate limits are someone else's problem. When you have 100+ users asking questions... Congratulations. You have invented a distributed systems problem. This was probably my favourite hack. Instead of relying on one API key, I created multiple API lanes. Something roughly like: Incoming Request | v LLM Gateway | +-----------+-----------+ | | Gemini lanes OpenRouter lanes G01...G10 N01...N10 | | +-----------+-----------+ | v Response I ended up with 20 lanes : Each lane had its own quota and concurrency control. The backend could distribute requests across them instead of hammering a single API key until it died. And because multiple users could hit the system simultaneously, I used PostgreSQL-based locking to coordinate access. Was this the most elegant infrastructure ever created? No. Was it considerably cheaper than paying for a giant API bill? Yes. At this point I had an AI system. Now I needed somewhere to run it. I looked at cloud hosting. Then I looked at the prices. Then I looked at my budget. Then I looked back at my laptop. We reached an agreement. My laptop was now the cloud. I ran the application locally and exposed it through ngrok At some point I stopped calling it a laptop and started calling it production infrastructure . It made me feel better. The system wasn't being built for me. It wasn't: "Look, here's my RAG demo." People were actually going to use it. Potentially 100+ people. And I didn't want the first question on event day to be: "Why is the website down?" So I added: I limited each team to 10 questions. This wasn't only about saving API calls—it also made the event more interesting because we could later look at the prompts participants were asking and evaluate how they were using the system. This was where the project became more interesting technically. I didn't just want: "Here are some chunks that look relevant." I wanted the system to actually reason about what kind of information it needed to retrieve. The system was running on my laptop. The laptop was connected to the internet. ngrok was exposing it. The APIs were running through free quotas. PostgreSQL was handling state. Qdrant was handling vector search. The RAG pipeline was running. And then... people started using it. Questions came in. Answers came back. People were actually using the system to search through the company information. And it worked. No paid cloud infrastructure. No expensive LLM subscription. No giant GPU server. No fancy production cluster. Just a lot of engineering, questionable financial decisions, and one laptop that was working much harder than it deserved to. The biggest lesson wasn't about RAG. It was about constraints . When you have unlimited resources, the solution is often: When you have ₹0, you can't say that. You have to ask: "Okay. What can I change instead?" And eventually, all those little problems became an actual system. That's probably the part of the project I'm most proud of. | Layer | Technologies | |---|---| | Frontend | React, Vite, TypeScript, Tailwind, PDF.js | | Backend | FastAPI, Uvicorn, Async REST APIs, SSE streaming | | AI & Retrieval | Gemini, OpenRouter / Nemotron, SentenceTransformers BAAI/bge-small-en-v1.5 , Two-stage retrieval, Reranking, Adaptive response depth | | Data & Storage | PostgreSQL, SQLAlchemy, Qdrant | | Infrastructure | Docker, Docker Compose, ngrok and my laptop | The complete project is open source: https://github.com/Pawan-19012006/Kairos-RAG-System https://github.com/Pawan-19012006/Kairos-RAG-System If I rebuilt this today, I'd spend much more time designing the evaluation and observability layer from the beginning. Getting a RAG system to answer is relatively easy. Getting it to answer correctly, consistently, with the right evidence, under real usage constraints is where the interesting engineering begins. And that's where this project stopped being "I built a chatbot" and became "I built a system that had to survive actual users." That's a much more fun problem. P.S. If you're building something similar and you're worried because you don't have a huge cloud budget: Don't let the lack of money decide whether the project is possible. Sometimes the most interesting engineering happens when you have no other option. And sometimes... your laptop becomes the cloud. 😅