{"slug": "gemini-reads-the-floor-plan-code-decides-what-it-means", "title": "Gemini reads the floor plan. Code decides what it means.", "summary": "A developer built a system that uses Gemini to analyze floor plans and satellite images for house hunting, separating perception from judgment to make results cacheable, reproducible, and accurate. The system scored 93% exact accuracy on generated floor plans, with the developer noting that a lighter model, gemini-3.5-flash-lite, performed equally well in 1.5 seconds, and that fallback chains are essential to handle Google's capacity issues.", "body_md": "*I created this post for the purposes of entering the All Things Agentic Hackathon.*\n\nMy wife and I have looked at twenty-two houses this year. We still haven't bought one.\n\nIt isn't that nothing's available. It's that what rules a house out for us isn't on Zillow. You can filter beds, baths and price. You can't filter which way the front door faces - and for my family, Vastu isn't negotiable. You can't filter \"is there a real bedroom downstairs with a full bath\" for my parents who are moving in. You can't filter whether the backyard is flat enough for a toddler, or whether the lot backs onto a four lane road.\n\nNone of that is a field in any listing feed. All of it is sitting in the floor plan and the satellite photo.\n\nSo it isn't a search problem. It's a seeing problem.\n\nI built something over a weekend that appeared to analyse floor plans. Scores, findings, confident write-ups about entrance orientation and yard grade.\n\nThen I read my own code. The image fetching was silently crashing. Every \"analysis\" was the model working from the street address alone which is exactly what I'd asked it to do. It had never been shown a single drawing.\n\nThat one's on me, and it's the reason the rest of this post exists.\n\nIt was also slow 30 seconds a scan and three of five aerial images were 404ing behind a loading spinner that never resolved.\n\nThe thing that made it work was deciding what the model is *for*.\n\n**Gemini does perception. Code does judgement.**\n\nThe model gets two images and one question: what is physically in this drawing? Where's the front door. Which quadrant is the kitchen. Does the downstairs bathroom have a tub in it, or just a toilet and a sink. It never sees my preferences and never produces a score.\n\nA scoring engine in TypeScript takes those findings and decides what they're worth *to me*.\n\nThat one separation bought four things I didn't expect:\n\n**It made results cacheable.** Perception depends only on the images, so it's computed once per house. Changing a preference rescores from cache in 23 milliseconds with zero model calls.\n\n**It made scores reproducible.** Same house, same number, every time. You can compare houses across weeks and the comparison means something.\n\n**It made the rulebook swappable.** Vastu and Feng Shui are lookup tables, not prompts. They genuinely disagree a south-facing entrance is a flaw in one and the classical ideal in the other so switching reranks the whole list without calling Gemini at all.\n\n**It made accuracy measurable.** Which turned out to matter most.\n\nI generate the floor plans with Gemini 3.1 Flash Image, from written specs. That sounds like a shortcut. It's the opposite.\n\nIf I'd scraped real listings, I'd have no answer key. I could show you a confident-looking output and neither of us could tell whether it was right.\n\nBecause I wrote the spec each drawing was made from, I know the correct answer for every one. So there's a harness that replays the agent's readings against that truth:\n\n```\nexact     39/42   93%\nadjacent   3/42    7%     (one compass point out on a hand-drawn plan)\nwrong      0/42    0%\n```\n\nNothing read backwards. And it runs offline in about a second, so anyone can rerun it.\n\nThe three \"adjacent\" misses are the model saying North where the spec said North-East. That's the disagreement two surveyors would have.\n\n** gemini-3.5-flash was the wrong default.** On identical two-image requests it once took\n\n`gemini-3.5-flash-lite`\n\nanswered the same request in 1.5 seconds at the same accuracy. Both satisfy the hackathon's model requirement. I now run a fallback chain and record which model produced each reading.**503 UNAVAILABLE is not 429 RESOURCE_EXHAUSTED.** I spent an evening assuming I was rate-limited. Every single failure was capacity on Google's side. Enabling billing doesn't fix it. A fallback chain does.\n\n**The Firestore client throws an uncaught exception when credentials are missing** — from a deferred gRPC stub, outside any try/catch around the call that triggered it. The process just dies seconds after boot. You have to check for credentials before constructing the client.\n\n**Cloud Run throttles CPU to near zero between requests.** OpenTelemetry's batch processor never fires its timer, so buffered spans die with the container. Export per-span instead.\n\nFeedback was supposed to be the good bit: say what you think in plain English, watch it rerank.\n\nIt didn't work. Rejecting a house moved its score by about a point. Sometimes *up*.\n\nThe maths was fine and the design was wrong. Feedback adjusted global weights, and a score is a weighted average over seven dimensions — so one weight moving 0.2 shifts the total by under a point. Worse, raising the weight of a dimension a house scores *well* on raises that house. I rejected the top-ranked property three times and watched it stay top-ranked.\n\nWeights express what you like in general. They cannot express *not this one*.\n\nThe fix was to record a verdict against the specific property. Reject a house and it drops to 25 and falls to the bottom, flagged. Global weight learning still happens beside it, it's just no longer asked to do a job it structurally can't.\n\nI'd been tuning numbers when the actual problem was a missing concept.\n\nIt runs unattended. Cloud Scheduler triggers it every morning; it reads whatever hit the market overnight and decides on its own whether anything is worth surfacing. Staying quiet is a valid outcome an agent that pings you about every new listing is a worse email alert.\n\nWhen you've picked what's worth seeing, it plans the day: orders the stops by geography, allocates time, says what to check at each door based on what it found in the plan, and hands back a Google Maps route.\n\nBuilt on Gemini 3.5 Flash-Lite through the Google GenAI SDK, running on Cloud Run with Firestore, Cloud Scheduler and Cloud Trace.\n\nCode: [https://github.com/skg0525/allthingsagentichackathon](https://github.com/skg0525/allthingsagentichackathon)\n\nI'm still house hunting. But I'm touring the right three instead of the wrong twenty-two.", "url": "https://wpnews.pro/news/gemini-reads-the-floor-plan-code-decides-what-it-means", "canonical_source": "https://dev.to/shivam_garg5/gemini-reads-the-floor-plan-code-decides-what-it-means-ojl", "published_at": "2026-08-31 04:22:36+00:00", "updated_at": "2026-08-31 04:51:35.167400+00:00", "lang": "en", "topics": ["artificial-intelligence", "computer-vision", "ai-tools", "ai-agents"], "entities": ["Gemini", "Google", "TypeScript", "Firestore"], "alternates": {"html": "https://wpnews.pro/news/gemini-reads-the-floor-plan-code-decides-what-it-means", "markdown": "https://wpnews.pro/news/gemini-reads-the-floor-plan-code-decides-what-it-means.md", "text": "https://wpnews.pro/news/gemini-reads-the-floor-plan-code-decides-what-it-means.txt", "jsonld": "https://wpnews.pro/news/gemini-reads-the-floor-plan-code-decides-what-it-means.jsonld"}}