{"slug": "dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-2", "title": "Dabba AI: Solving the 8 PM \"Aaj Khane Mein Kya Banega?\" Crisis with Local Vision & Gemma 2", "summary": "A developer built Dabba AI, a Streamlit app that uses locally hosted Gemma 3 vision and text models via Ollama to identify raw ingredients from a fridge photo and generate a five-step Indian tiffin recipe, with ElevenLabs providing hands-free audio narration. The project constrains the vision model to comma-separated ingredient extraction and the text model to a home-cook persona limited to standard Indian spices, avoiding paid cloud vision APIs and recurring token costs.", "body_md": "*This is a submission for the [Hacktoberfest Weekend Challenge: Build for a Friend](https://dev.to/challenges/hacktoberfest-weekend-2026-10-01)*\n\nEvery evening around 8:00 PM, my flatmate stands in front of our open refrigerator door for five full minutes, blankly staring at a random collection of half-used vegetables, leftover paneer, and expired hopes.\n\nThe universal Indian bachelor dilemma: \"Aaj khane mein kya banega?\" (What should we cook today?)\n\nMost cooking apps assume you have 25 exotic ingredients or want a generic 40-minute chef recipe. Food delivery apps just want you to give up and order biryani again.\n\nI built Dabba AI for him.\n\nYou snap a quick photo of whatever raw ingredients are sitting in your fridge or kitchen counter, pick your dietary preference (Jain, High Protein, Vegan, or under 15 minutes), and hit generate.\n\nIn seconds:\n\nGemma3 runs locally to inspect the photo and extract only the raw ingredients.\n\nGemma 3 takes those ingredients and acts like an Indian mom/home cook—crafting a quick, realistic 5-step dabba (tiffin) recipe using standard Indian spice-box staples (haldi, jeera, rai, hing, dhania, etc.).\n\nElevenLabs reads the recipe aloud in the background so you can cook hands-free with oily or turmeric-stained fingers without touching your laptop screen.\n\nNo paid cloud vision APIs. No monthly API billing surprises. The vision and recipe intelligence stay right on the laptop via Ollama.\n\nPull the Ollama models:\n\nBash\n\n`ollama pull gemma3`\n\nClone and install dependencies:\n\n`git clone https://github.com/your-username/dabba-ai.git cd dabba-ai pip install streamlit ollama elevenlabs`\n\nLaunch the Streamlit app:\n\n`export ELEVENLABS_API_KEY=\"your-api-key\" streamlit run app_gemma.py`\n\nThe stack is intentional: zero bloated cloud microservices for the core AI logic.\n\n```\n   [ Fridge Photo ] \n          │\n          ▼\n┌───────────────────┐\n│     Gemma3 Vision │  (Local via Ollama: Extracts raw ingredients)\n└─────────┬─────────┘\n          │ comma-separated tokens\n          ▼\n┌───────────────────┐\n│   Gemma 3 Model   │  (Local via Ollama: Indian home-cook persona)\n└─────────┬─────────┘\n          │ clean, 5-step recipe text\n          ▼\n┌───────────────────┐\n│ ElevenLabs Audio  │  (Hands-free kitchen voice assistant)\n└───────────────────┘\n```\n\nI used Gemma3 served locally via Ollama. The prompt is intentionally constrained:\n\nPython\n\n`vision_prompt = (\n\n    \"Identify all the raw food ingredients visible in this image. \"\n\n    \"List them concisely as comma-separated words only. No sentences, no explanations.\"\n\n)\n\nvision_response = ollama.chat(\n\n    model='paligemma',\n\n    messages=[{\n\n        'role': 'user',\n\n        'content': vision_prompt,\n\n        'images': [uploaded_file.getvalue()]\n\n    }]\n\n)\n\ningredients = vision_response['message']['content'].strip()`\n\nPassing strict extraction instructions prevents the vision model from hallucinating cooking steps early—it acts strictly as an ingredient spotter.\n\nGeneric LLMs tend to suggest ingredients you don't have: \"Add 2 tbsp of sour cream and fresh tarragon.\" That fails the Indian kitchen test.\n\nI constrained Gemma 3 to act like a practical home cook: it is strictly allowed to use the identified ingredients plus standard spices from a traditional Indian masala dabba (haldi, jeera, rai, hing, red chilli, salt).\n\nPython\n\n`text_prompt = f\"\"\"You are a creative Indian home cook. I have these ingredients: {ingredients}.\n\nCreate a fast, practical dabba/tiffin recipe using ONLY these ingredients and standard Indian spices (salt, haldi, jeera, rai, hing, dhania powder, red chilli powder).\n\nDietary constraint: {dietary_need}.`\n\nRules:\n\nWith the toggle enabled, the recipe text streams directly to ElevenLabs' eleven_multilingual_v2 model, producing natural spoken instructions right from the Streamlit UI.\n\nWhy Open-Weight Models Matter Here\n\nIf you build an app that runs every day in your home, relying entirely on paid frontier APIs creates recurring friction. You worry about token usage, billing tiers, or whether an endpoint is down when you are just trying to feed yourself before a late-night coding session.\n\nRunning Gemma3 and Gemma 3 locally means:\n\nZero inference cost: We can snap 10 different angles of the pantry shelf and test recipes without spending a cent.\n\nPrivacy by default: Photos of personal living spaces and fridges stay entirely local on your machine.\n\nSpeed & Autonomy: No rate limits, no waiting in cloud queues when dinner needs to be made.\n\nHow to Run It Locally\n\nSince this runs on local weights rather than a hosted cloud server, you can spin it up on your own machine in 3 steps:\n\nBash\n\n`ollama pull gemma3`\n\nClone and install dependencies:\n\nBash\n\n`git clone https://github.com/your-username/dabba-ai.git`\n\ncd dabba-ai\n\npip install streamlit ollama elevenlabs\n\nLaunch the Streamlit app:\n\nBash\n\n`export ELEVENLABS_API_KEY=\"your-api-key\"`\n\nstreamlit run app_gemma.py\n\nNo more staring into the fridge void. Snap, cook, eat.``\n\nOpen innovation is what turns AI from an expensive subscription product into everyday household plumbing. When you build utility tools for simple daily friction—like deciding what to cook dinner with—relying entirely on closed, metered APIs breaks the whole premise:\n\nNo Billing Anxiety for Daily Living: If snapping a photo of your vegetable tray costs a fraction of a dollar in token fees, you will instinctively avoid using the tool. With open-weight models like PaliGemma and Gemma 2 via Ollama, inference costs nothing once downloaded. You can snap four different angles of a half-empty crisper drawer without watching an API billing counter tick up.\n\nPrivacy Inside Personal Living Spaces: Photos of your kitchen counter, fridge shelves, and home surroundings are inherently personal. Open innovation allows multi-modal vision to execute directly on local silicon, ensuring candid photos never get transmitted to third-party cloud servers or stored in model training logs.\n\nResilience and True Offline Autonomy: Cooking happens when you are hungry, not when an upstream cloud status page is operational. Open weights give developers complete control over execution, latency, and system dependencies, so the tool stays dependable regardless of network drops or API deprecations.\n\nBest Gemma Application: For designing a multi-step pipeline powered locally by Google's open models—using Gemma3 for multi-modal ingredient extraction and Gemma 3 for constrained culinary reasoning.\n\nBest Audio / Speech Integration: For using ElevenLabs (eleven_multilingual_v2) to turn structured recipes into hands-free spoken audio so the cook never has to smudge a screen with spice-covered hands.", "url": "https://wpnews.pro/news/dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-2", "canonical_source": "https://dev.to/shreyashbhor2006/dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-gemma-2-474b", "published_at": "2026-10-03 17:55:22+00:00", "updated_at": "2026-10-03 18:07:59.292597+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "computer-vision", "ai-tools"], "entities": ["Dabba AI", "Gemma 3", "Ollama", "ElevenLabs", "Streamlit", "Hacktoberfest"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-2", "markdown": "https://wpnews.pro/news/dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-2.md", "text": "https://wpnews.pro/news/dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-2.txt", "jsonld": "https://wpnews.pro/news/dabba-ai-solving-the-8-pm-aaj-khane-mein-kya-banega-crisis-with-local-vision-2.jsonld"}}