Gemini Vision can actually tell if your dog is judging you

GeminiVision to analyze dog expressions and return a "judgment percentage." It's a goofy idea, but the technical implementation was a great way to test out some specific AI workflow patterns, especially regarding structured data and deployment.

The core of the app is a SvelteKit 5 frontend (using the new runes API) and Tailwind CSS 4, hosted on Netlify. The main goal wasn't just to make a prompt that works, but to build a production-ready tool that doesn't crash or leak user data.

The technical breakdown #

The most important part of the AI workflow here is how the image is handled before it even hits the LLM. I implemented a client-side pipeline where the photo is drawn to a canvas and re-exported as a compressed JPEG. This serves two purposes: it keeps the payload small for faster API responses and automatically strips EXIF metadata (like GPS coordinates), which is a must for any real-world deployment.

For the backend, I'm using a SvelteKit API route that hits gemini-2.5-flash

. The "secret sauce" for reliability here is the responseSchema

. Instead of the usual "please return JSON" prompt—which often fails or includes annoying markdown backticks—the schema forces the model to adhere to a strict structure.

Here is the basic logic for the API call:

// Simplified logic for the /api/judge route
const result = await model.generateContent({
  contents: [{ role: 'user', parts: [imagePart, textPart] }],
  generationConfig: {
    responseMimeType: 'application/json',
    responseSchema: {
      type: 'object',
      properties: {
        judgmentLevel: { type: 'number' },
        emotion: { type: 'string' },
        innerMonologue: { type: 'string' },
        advice: { type: 'string' },
        breedGuess: { type: 'string' },
      },
      required: ['judgmentLevel', 'emotion', 'innerMonologue', 'advice', 'breedGuess'],
    },
  },
});

Handling scale and stability #

Since this is a public-facing app, I couldn't just leave the API open. I integrated Upstash Redis to handle rate limiting using a sliding-window approach (capped at 5 requests per hour per IP). This is way more reliable than in-memory limiting because it persists across serverless cold starts.

I also added several "boring" but critical features to move this from a prototype to a real app:

Input Validation: Strict checks on file size and MIME types.Timeouts: The Gemini call has a hard timeout to prevent hanging requests.Error States: Proper frontend feedback if the AI service is down or the API key hits a limit.

The final polish includes

canvas-confetti

that triggers based on the judgment score and a dynamic prompt that uses navigator.language

to generate the dog's "inner monologue" in the user's native tongue. It's a simple project, but it proves how powerful combining a vision model with a strict schema can be for building lightweight, interactive AI tools.Next Using Vuepress to build a community recipe book is a great way →