This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend I built NutriScan.AI for my close friend Dave, an undergraduate university student who set an earnest personal goal to reduce his weight from 82.4 kg down to a healthy, lean 75.0 kg (165.3 lbs) before midterm exams.
Managing weight loss on a modern college campus sounds simple on paper, but when I watched Dave try to do it, I realized the current software ecosystem failed him at every turn:
I built NutriScan.AI as a sovereign, mathematically grounded nutrition studio that solves all four dilemmas. It combines multimodal computer vision with an offline USDA FoodData Central database, dynamic metabolic forecasting via Prior Labs' TabPFN tabular foundation model, and an AI voice coach that gives him an audio debrief on his walk across campus.
**Live Production Application**: [https://nutriscan-ai-fwn8.onrender.com/](https://nutriscan-ai-fwn8.onrender.com/)
**Mobile & Desktop Interactive Studio**: [https://nutriscan-ai-fwn8.onrender.com/scan](https://nutriscan-ai-fwn8.onrender.com/scan)
**Video Demo**: [Watch the Full Walkthrough on GitHub](https://github.com/user-attachments/assets/00fd8620-69ec-42ae-8bef-5e0c47a6debe)
+250ml / +500ml), and a 16:8 intermittent fasting timer.
NutriScan AI eliminates the "Calorie Guesswork Crisis", the dangerous "AI Vision Hallucination Trap", and the "Static Math Fallacy". Commercial food tracking applications charge $35/month while sending intimate meal photos and biological logs to centralized corporate clouds, all while relying on 1990s static formulas (like Mifflin-St Jeor) that completely fail to account for metabolic adaptation, non-exercise activity thermogenesis (NEAT), and water retention. NutriScan AI pairs open-weight Google Gemma 2 Vision with 100% deterministic USDA FoodData Central grounding and Prior Labs' TabPFN—the world's leading tabular foundation model—to discover a user's true dynamic daily energy expenditure (TDEE) and project their exact 28-day weight trajectory with Bayesian confidence intervals.
A real-device walkthrough of the NutriScan.AI companion, demonstrating live camera scanning, deterministic USDA FoodData Central macronutrient grounding, TabPFN dynamic metabolic forecasting, and…
When I sat down with Dave to design NutriScan.AI, I established a strict architectural invariant:
Large Language Models must NEVER do calorie math.
Computer vision must ONLY segment food portions in grams.
USDA tables must compute deterministic arithmetic.
Tabular foundation models must forecast human metabolism.
I architected the platform around five core open-source AI and engineering pillars:
I used open-weight vision models (running locally via Ollama or hosted) strictly for geometric scene decomposition: identifying food boundaries, item classes, and portion volume estimations in grams without routing private meal photos to commercial ad networks.
Instead of letting an AI guess nutritional values, I mapped Gemma's food detections to an offline slice of USDA FoodData Central. The system matches recognized items against verified reference IDs (FDC IDs) and multiplies nutritional density by the portion gram weight using pure Python arithmetic:
$$\text{Nutrient}{\text{total}} = \sum{i=1}^{N} \left( \frac{\text{Portion Grams}_i}{100} \times \text{USDA Density per 100g}_i \right)$$ Zero generative hallucinations. 100% verifiable clinical truth.
This is the technological crown jewel of NutriScan.AI. Rather than forcing Dave onto static 1990 population averages, TabPFN ingests Dave's rolling 30-day biological check-in dataset (dave_metabolic_log.csv tracking daily calories, protein, carbs, fat, campus step count, sleep hours, and morning scale weight). In a single forward pass without backpropagation loops or fine-tuning TabPFN evaluated Dave's non-linear weight changes and discovered that his true dynamic expenditure was 3,152 kcal/day nearly 600 kcal higher than textbook formulas predicted! It then outputs a 28-day Bayesian trajectory forecast complete with confidence envelopes.
To keep Dave motivated on his walk across campus, I integrated ElevenLabs' neural text-to-speech API (Rachel voice model) to synthesize an intelligent 30-second audio debrief analyzing his daily macros and celebrating milestone achievements.
Every pipeline stage from visual inference to TabPFN in-context evaluation and ElevenLabs audio buffers—is instrumented with distributed tracing to guarantee sub-second reliability.
Open innovation is the entire reason NutriScan.AI exists. Here is why open models made possible what closed APIs never could:
When I deployed the web application to Render and handed Dave the phone over a quick dining hall lunch, his reaction was instantaneous:
"Are you serious? You built this in a weekend? The camera scanner picked up my cafeteria grilled chicken and rice bowl instantly, and the calories match the USDA label to the gram. But the crazy part is the 28-Day Trajectory chart: when I dragged the slider to 2,100 kcal, it told me I'd reach 75.0 kg on Day 21 fitting right before midterm week. And the voice debrief on my walk back to the dorm literally sounded like a personal fitness coach in my pocket!"
I built and iterated on NutriScan.AI with my coding assistant using advanced agentic AI pair programming. You can explore the complete, interactive development session below:
(If the interactive widget doesn't load in your browser, you can also view the session transcript directly at dev.to/agent_sessions/nutriscanai-full-end-to-end-build-for-hacktoberfest-build-for-a-friend-baxn6p)
I am entering NutriScan.AI into the following partner categories:
https://nutriscan-ai-fwn8.onrender.com), serving both the responsive web studio frontend and the asynchronous AI endpoints with sub-second response times.dave_metabolic_log.csv) via in-context learning. TabPFN discovered Dave's true dynamic TDEE (3,152 kcal) and predicted his 28-day Bayesian weight trajectory with 95% confidence intervals.Rachel voice model) to give the application an auditory coaching personality, delivering an empathetic 30-second metabolic debrief directly to Dave's headphones as he walks across campus.app/services/tracing.py), monitoring latency, execution performance, and failure boundaries.