{"slug": "100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos", "title": "100+ Users, Free API Keys and ₹0 Cloud Budget: How I Built Kairos", "summary": "A developer built Kairos, a retrieval-augmented generation (RAG) question-answering system for a departmental event, serving over 100 users on a ₹0 infrastructure budget. The system routes requests across 20 API lanes using free Gemini and OpenRouter/Nemotron tiers with PostgreSQL-based locking, and runs locally on a laptop exposed via ngrok, with Qdrant handling vector search and each team limited to 10 questions.", "body_md": "I wanted to build a simple RAG system.\n\nThat sentence aged badly.\n\nThe original problem was actually pretty simple.\n\nWe were organizing a departmental event called **Kairos**, where participants had to study company information and create pitch decks.\n\nThe organizers had a lot of company reports.\n\nThe participants had to read them.\n\nAnd the obvious solution was:\n\n**Print everything.**\n\nExcept there was one tiny problem.\n\nThere were **a lot of pages.**\n\nAnd there were **a lot of participants.**\n\nPrinting enough copies for everyone wasn't exactly friendly to the budget.\n\nSo I had a stupidly simple thought:\n\n*Why don't we just put all the documents into an AI system and let participants ask questions?*\n\nAnd that's how I accidentally created a much bigger problem for myself.\n\nIf you've built RAG systems before, you know how innocent that sentence sounds.\n\nUpload PDFs → chunk them → embed them → retrieve relevant chunks → send them to an LLM.\n\nEasy.\n\nUntil you actually try to make people use it.\n\nI needed:\n\nAnd I had one major constraint:\n\nNot \"small startup budget.\"\n\nI mean **₹0 infrastructure budget**.\n\nThe next question was obvious:\n\n**Which LLM should I use?**\n\nPaid APIs?\n\nNo.\n\nI didn't have money for that.\n\nSo I started looking at free API tiers.\n\nI ended up using **Gemini** as the primary model and **OpenRouter/Nemotron** as a fallback.\n\nProblem solved?\n\nAbsolutely not.\n\nFree APIs have limits.\n\nAnd when you have one user asking a question, rate limits are someone else's problem.\n\nWhen you have 100+ users asking questions...\n\n**Congratulations. You have invented a distributed systems problem.**\n\nThis was probably my favourite hack.\n\nInstead of relying on one API key, I created multiple API lanes.\n\nSomething roughly like:\n\n```\n         Incoming Request\n                |\n                v\n         LLM Gateway\n                |\n    +-----------+-----------+\n    |                       |\nGemini lanes          OpenRouter lanes\n G01...G10               N01...N10\n    |                       |\n    +-----------+-----------+\n                |\n                v\n             Response\n```\n\nI ended up with **20 lanes**:\n\nEach lane had its own quota and concurrency control. The backend could distribute requests across them instead of hammering a single API key until it died. And because multiple users could hit the system simultaneously, I used PostgreSQL-based locking to coordinate access.\n\nWas this the most elegant infrastructure ever created? No. Was it considerably cheaper than paying for a giant API bill? Yes.\n\nAt this point I had an AI system. Now I needed somewhere to run it.\n\nI looked at cloud hosting. Then I looked at the prices. Then I looked at my budget. Then I looked back at my laptop.\n\nWe reached an agreement. **My laptop was now the cloud.**\n\nI ran the application locally and exposed it through ngrok\n\nAt some point I stopped calling it a laptop and started calling it *production infrastructure*. It made me feel better.\n\nThe system wasn't being built for me. It wasn't: *\"Look, here's my RAG demo.\"* People were actually going to use it. Potentially 100+ people. And I didn't want the first question on event day to be: *\"Why is the website down?\"*\n\nSo I added:\n\nI limited each team to 10 questions. This wasn't only about saving API calls—it also made the event more interesting because we could later look at the prompts participants were asking and evaluate how they were using the system.\n\nThis was where the project became more interesting technically. I didn't just want: *\"Here are some chunks that look relevant.\"* I wanted the system to actually reason about what kind of information it needed to retrieve.\n\nThe system was running on my laptop. The laptop was connected to the internet. ngrok was exposing it. The APIs were running through free quotas. PostgreSQL was handling state. Qdrant was handling vector search. The RAG pipeline was running.\n\nAnd then... people started using it. Questions came in. Answers came back. People were actually using the system to search through the company information.\n\n**And it worked.**\n\nNo paid cloud infrastructure. No expensive LLM subscription. No giant GPU server. No fancy production cluster. Just a lot of engineering, questionable financial decisions, and one laptop that was working much harder than it deserved to.\n\nThe biggest lesson wasn't about RAG. It was about **constraints**.\n\nWhen you have unlimited resources, the solution is often:\n\nWhen you have ₹0, you can't say that. You have to ask: **\"Okay. What can I change instead?\"**\n\nAnd eventually, all those little problems became an actual system. That's probably the part of the project I'm most proud of.\n\n| Layer | Technologies | \n|---|---|\n| **Frontend** | React, Vite, TypeScript, Tailwind, PDF.js | \n| **Backend** | FastAPI, Uvicorn, Async REST APIs, SSE streaming | \n| **AI & Retrieval** | Gemini, OpenRouter / Nemotron, SentenceTransformers ( `BAAI/bge-small-en-v1.5` ), Two-stage retrieval, Reranking, Adaptive response depth | \n| **Data & Storage** | PostgreSQL, SQLAlchemy, Qdrant | \n| **Infrastructure** | Docker, Docker Compose, ngrok (and my laptop) | \n\nThe complete project is open source:\n\n[https://github.com/Pawan-19012006/Kairos-RAG-System](https://github.com/Pawan-19012006/Kairos-RAG-System)\n\nIf I rebuilt this today, I'd spend much more time designing the **evaluation and observability layer** from the beginning. \n\nGetting a RAG system to answer is relatively easy. Getting it to answer correctly, consistently, with the right evidence, under real usage constraints is where the interesting engineering begins.\n\nAnd that's where this project stopped being *\"I built a chatbot\"* and became **\"I built a system that had to survive actual users.\"**\n\nThat's a much more fun problem.\n\n**P.S.** If you're building something similar and you're worried because you don't have a huge cloud budget: Don't let the lack of money decide whether the project is possible. Sometimes the most interesting engineering happens when you have no other option. \n\nAnd sometimes... your laptop becomes the cloud. 😅", "url": "https://wpnews.pro/news/100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos", "canonical_source": "https://dev.to/pawan_19012006/100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos-25k5", "published_at": "2026-09-13 21:54:29+00:00", "updated_at": "2026-09-13 22:21:27.228968+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-infrastructure", "ai-products", "mlops"], "entities": ["Kairos", "Gemini", "OpenRouter", "Nemotron", "PostgreSQL", "Qdrant", "ngrok"], "alternates": {"html": "https://wpnews.pro/news/100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos", "markdown": "https://wpnews.pro/news/100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos.md", "text": "https://wpnews.pro/news/100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos.txt", "jsonld": "https://wpnews.pro/news/100-users-free-api-keys-and-0-cloud-budget-how-i-built-kairos.jsonld"}}