This is a submission for Weekend Challenge: Dog Days Edition
Pawscript. Upload a photo of your dog, optionally with a bark, whine, or sigh, and Gemini reads the actual photo (and sound, if given) and writes a short, warm letter in your dog's own voice, along with a few honest "Tail Tell" observations about their visible posture and expression.
My goal was to avoid the usual "upload a photo → get a random label" pattern that many pet apps use, and instead build something that is actually grounded in the specific photo you provide. So if you scan the same dog twice, the result can genuinely be different rather than feeling like a copy-paste response.
Live: https://pawscript.ai.studio I started with a UI I liked, a five-paw "scanning" progress meter with a breathing/pulse animation, and rebuilt the logic underneath so it actually does something.
The core is a single multimodal call to Gemini. The uploaded photo, along with optional audio, is sent with a system instruction that tells the model to reference specific visual details it can actually see in that photo, such as the coat, posture, setting, and expression. This helps keep the letters specific to the image instead of making them sound generic. I also enforced structured JSON output with letter
, mood_tag
, observations
, and used_audio
, so the UI never has to guess how to parse the response.
The trickiest part was the optional audio path. I had to send the image and inline base64 audio in the same request and ask Gemini to combine the acoustic tone, such as a bright yip or a low grumble, with the visual details when writing the letter. I also made sure it never invents sound characteristics when no audio is provided.
I tied the paw-progress animation to the real fetch lifecycle instead of using a fixed timer, so the scanning moment is based on the actual request rather than being purely decorative.
If shipped, letters can optionally be read aloud using ElevenLabs voice narration. Nothing uploaded is stored. Photos and audio go directly to Gemini for analysis and aren't kept.
Best Use of Google AI — Gemini multimodal vision + audio understanding with enforced structured output.
Best Use of ElevenLabs — if you shipped voice narration, keep this line; if not, delete it.
Thanks for reading, and happy Dog Days! 🐾