This is a submission for Weekend Challenge: Dog Days Edition
DogeVault Stylist is a Next.js app that turns a single photo of your dog into a full shopping experience. You upload a dog photo, and the app:
The goal was to build something that feels like a real product — not a demo with hardcoded data. Every product card links to a live listing. Every transaction hits Solana devnet. Every voice line is synthesized.
[Video demo placeholder — upload and link your demo video here]
AI dog stylist + autonomous shopper for the Dog Days hackathon. Upload a dog photo, get a YouCam studio portrait, hear an ElevenLabs stylist reaction, fetch real products through Apify MCP, and approve a budget-limited USDC transfer on Solana devnet.
apps/web
)programs/doge_vault
)apify/e-commerce-scraping-tool
apps/web Next.js app
src/lib Server/client libraries
src/app/api/agent Upload -> YouCam -> Gemini -> Apify orchestration
src/app/api/voice ElevenLabs text-to-speech
src/components DogeVault UI and Solana wallet flows
programs/doge_vault Anchor vault program
PLAN.md Full hackathon plan and decisions
Copy .env.example
to .env
and fill in keys.
Important values already configured for devnet:
The repo is a pnpm monorepo:
apps/web
— Next.js 16 frontend and API routesprograms/doge_vault
— Anchor 0.32.1 Solana programadk-apify-sample/
— Standalone Google ADK agent (Python)Everything starts with one API call to /api/agent
. The server-side orchestrator chains six providers in sequence, streaming progress back to the UI via SSE:
Upload → YouCam enhance → YouCam background replace → Gemini profile
→ Apify product search → Virtual try-on → ElevenLabs voice
Each step is isolated in its own module under apps/web/src/lib/
, so a missing API key degrades gracefully instead of crashing the whole pipeline. The UI shows a status badge per provider (ok
/ missing_key
/ error
).
This was the hardest part to get right. The Apify MCP server solved my product extraction problem. It exposes Apify Actors as tools over the Model Context Protocol. I call the apify/e-commerce-scraping-tool
Actor from my Next.js backend using the official MCP TypeScript SDK:
const client = new Client({ name: "dogevault-stylist", version: "0.1.0" });
const transport = new StreamableHTTPClientTransport(
new URL("https://mcp.apify.com/?tools=apify/e-commerce-scraping-tool,get-actor-run,get-dataset-items"),
{ requestInit: { headers: { Authorization: `Bearer ${APIFY_TOKEN}` } } }
);
await client.connect(transport);
Actor runs are async — the server returns a runId
, and I poll get-actor-run
until SUCCEEDED
, then read get-dataset-items
. The whole flow returns normalized product cards with images, prices, ratings, and direct URLs from Amazon, eBay, and other marketplaces.
Gemini handles three jobs:
The profile also tags the subject as dog
or human
, which determines which virtual try-on path the app takes downstream.
YouCam's virtual try-on API is human-only (I just knew about it during development) — it works great for clothes, hats, and shoes on people, but can't dress a dog. So I built a hybrid:
cloth-v3
, hat
, shoes
) with the Apify product image as the garment reference.qwen-image-3.0-pro
) edits the dog portrait to wear the selected product, using both the dog photo and product image as references.The stylist script from Gemini gets sent to ElevenLabs for text-to-speech. One gotcha: free ElevenLabs plans can't use library voices via the API (returns HTTP 402 paid_plan_required
). The app catches this and falls back to the browser's built-in SpeechSynthesis
— so the feature still works, just with a less polished voice.
The on-chain piece is an Anchor program that acts as a spending guardrail:
initialize_vault
approve_spend
This means even if the AI recommends a $500 Gucci dog collar, the vault won't let it through. After approval, the app mints a compressed NFT (cNFT) via Metaplex Bubblegum V2 as a receipt — containing the portrait, try-on image, product details, and transaction reference.
Best Use of ElevenLabs — The AI stylist's reaction script (generated by Gemini) is converted to speech via the ElevenLabs text-to-speech API. The app handles the free-plan 402 error gracefully, falling back to browser SpeechSynthesis
so the voice feature always works.
Best Use of Google AI — Gemini powers three core features: dog breed/style profiling from the uploaded photo, stylist script generation for the voice reaction, and product search query planning that feeds the Apify scraper. Gemini is the decision-making layer that ties the entire pipeline together.