CanineWhisperer

The CanineWhisperer project, developed by Zenieverse, is a multimodal veterinary ethology platform that uses Gemini Vision AI to decode canine body language, acoustic analysis to translate barks and whines, and ElevenLabs neural audio to give dogs a synthesized 'inner voice.' The system also includes ultrasonic whistle generation, Snowflake Cortex ML analytics for behavioral forecasting, and Solana-based digital passports with cNFT credentials, aiming to reduce shelter surrenders by providing real-time behavioral guidance to pet parents and trainers.

Overview & Purpose Canine AI Whisperer is an intelligent multimodal veterinary ethology and behavioral intelligence platform designed to bridge the communication gap between dogs and their humans. Our core goal is to transform modern canine care by translating subtle physical micro-signals, acoustic vocalizations, and behavioral telemetry into actionable, real-time guidance—preventing behavioral escalation and strengthening the bond between pet parents and their dogs. Key Capabilities & Architecture Multimodal Visual Posture Decoder Gemini Vision AI Analyzes real-time camera streams or uploaded photos to detect subtle body language cues ear carriage, commissure tension, tail angles, pupil dilation, and weight distribution . Generates instantaneous ethological diagnoses, arousal scores 0–100 , and step-by-step de-escalation action plans. Acoustic Bark Spectrogram & Translation Captures live canine vocalizations to extract fundamental frequency harmonics Hz , sound pressure intensity dB , and temporal cadence. Accurately classifies barks, whines, growls, and howls into emotional motivations e.g., territorial alert, separation distress, predatory excitement with human-language translations. Canine Voice Synthesis ElevenLabs Neural Audio Gives dogs their own distinctive "inner voice" based on tailored ethological personas e.g., The Hyperactive Herder, The Philosophical Frenchie, The Regal Retriever . Generates spoken translations and calming vocal cues using custom neural text-to-speech. Ultrasonic Whistle & Restorative Sound Studio Features a Web Audio tone generator capable of transmitting silent ultrasonic frequencies up to 22,000+ Hz for immediate recall and attention redirection without human disruption. Includes restorative harmonic frequencies 432Hz delta calm, 396Hz distress release, and 60 BPM maternal heartbeat loops for crate conditioning and thunderstorm anxiety. Snowflake Data Cloud & Cortex ML Analytics Simulates an enterprise-grade pet telemetry pipeline with millions of behavioral records across 80+ breeds. Utilizes Snowflake Cortex ML forecasting to highlight hourly reactivity spikes, trigger distributions, and breed-specific settle times, supported by a full SQL querying interface. Solana Canine Digital Passport & Micro-Economy Issues immutable on-chain pet identities cNFTs containing pedigree vitals, microchip SHA-256 signatures, and verifiable Canine Good Citizen CGC credentials. Integrates positive-reinforcement TREATS micro-rewards for completing behavioral milestones. Interactive Whisperer Coaching Dialogue Provides on-demand behavioral coaching grounded in balanced pack leadership and desensitization principles Exercise, Discipline, Affection for separation anxiety, resource guarding, and leash reactivity. Intended Goal Our vision is to empower dog parents, animal shelters, and trainers with accessible, scientific tools that reduce shelter surrenders caused by preventable behavioral issues, promote compassionate leadership, and foster happier, calmer pets. https://ai.studio/apps/6def5058-b655-4230-876d-2c8928ed8d6f https://ai.studio/apps/6def5058-b655-4230-876d-2c8928ed8d6f https://github.com/Zenieverse/CanineWhisperer https://github.com/Zenieverse/CanineWhisperer code Code ┌─────────────────────────────────────────────────────────────┐ │ CLIENT LAYER React 19 + Vite │ │ • WebRTC Live Camera Stream & HTML5 AudioContext Capture │ │ • Web Audio FFT Spectrogram & Dual-Frequency Oscillator │ │ • Recharts Interactive Telemetry & Responsive UI │ └──────────────────────────────┬──────────────────────────────┘ │ REST API / Async IPC ▼ ┌─────────────────────────────────────────────────────────────┐ │ BACKEND LAYER Node.js + Express │ │ • Ingestion Gateway & Structured Payload Verification │ │ • Multi-Model Prompt Engineering & Telemetry Pipeline │ └──────┬───────────────────────┬───────────────────────┬──────┘ │ │ │ ▼ ▼ ▼ ┌──────────────────────┐ ┌───────────────────┐ ┌───────────────────┐ │ GOOGLE GEMINI │ │ SNOWFLAKE │ │ SOLANA │ │ Multimodal Ethology │ │ Cortex ML Predict │ │ cNFT Identity DW │ └──────────────────────┘ └───────────────────┘ └───────────────────┘ A. Google Gemini API Multimodal Vision & Conversational Ethology Sub-Second Visual Biometrics: Rather than treating canine posture as simple object detection, we engineered clinical prompt schemas for Gemini 2.5 Flash Vision. The model systematically analyzes 6 distinct anatomical micro-markers: Ear Base Tension flattened vs. pricked vs. neutral Commissure Lip Retraction long loose pant vs. tight stress grimace Sclera Exposure "Whale eye" detection Spinal Rigidity & Weight Distribution forward territorial lean vs. rear avoidance load Pupil Dilation & Brow Furrowing Tail Carriage Angle and Oscillation Stiffness Structured Diagnostic Outputs: Outputs an Arousal Score , Cortisol Risk Tier, Immediate De-escalation Protocol, and a contextual inner monologue tailored to the pet’s breed pedigree. Conversational Coaching: Multi-turn dialogue coach grounded in the Cesar Millan pack leadership framework Exercise, Discipline, Affection in strict priority order . B. Snowflake Data Cloud & Cortex ML Analytics Enterprise Pet Telemetry Pipeline: Built a high-throughput event ingestion architecture CANINE TELEMETRY.BEHAVIOR LOGS modeling over historical records across 80+ breeds. Snowflake Cortex ML Forecasting: Utilizes SNOWFLAKE.ML.FORECAST and SNOWFLAKE.ML.TOP TRIGGERS algorithms to project hourly barking probability curves, quantify breed-specific settle times, and correlate weather barometric drops with anxiety spikes. Interactive SQL Studio: Embedded a query execution engine with pre-configured ethological queries, sub-millisecond execution simulations, and real-time schema filtering. C. ElevenLabs Neural Audio & Web Audio DSP Canine Voice Personas: Synthesizes the translated inner monologues into expressive, character-accurate speech with adaptive emotional inflections across 6 archetypes The Golden Goof, The Bulldog Lord, The Shepherd Guardian, The Frenchie Sassy, The Husky Dramatic, The Whisperer Sage . Web Audio Ultrasonic Synthesizer: Implemented zero-dependency AudioContext and OscillatorNode engines capable of generating precision frequencies from to true ultrasonic levels for silent recall, alongside calibrated and harmonic resonance tracks. D. Solana Blockchain Identity & Micro-Economy Verifiable cNFT Passports: Mints tamper-proof digital pet credentials encoding microchip SHA-256 hashes, pedigree vitals, and AKC/Canine Good Citizen CGC certificates verified on-chain. TREATS Token Economy: Implemented a positive-reinforcement micro-reward engine with interactive milestone verification and Devnet faucet replenishment. Decision: Process acoustic decibels and FFT spectral waveforms directly inside the browser using AnalyserNode.getByteFrequencyData before sending audio signatures to the server. Result: Zero latency in visual feedback and immediate ultrasonic counter-frequency playback during acute barking episodes. Graceful Multi-Tier Audio Fallback: Decision: Implemented an intelligent audio fallback matrix. When an external ElevenLabs API key is absent or rate-limited, the system seamlessly transitions to Web Speech Synthesis with dynamic pitch adjustment deep guard tone to puppy pitch , ensuring a consistent user experience. Unified Cross-Module State Synchronization: Decision: Built a reactive profile and telemetry state loop. Whenever a dog's profile is updated or a new photo/bark is decoded: The visual posture diagnosis updates. The telemetry record streams to the Snowflake event table. The Solana passport refreshes its verifiable traits and awards TREATS tokens. Editorial Aesthetic Design System: Decision: Rejected standard dark-mode templates in favor of a typography-led, warm ivory FAF9F6 and ink 1A1A1A aesthetic. Result: Clean, high-contrast readability across clinical diagnostic charts, spectrograms, and data grids without visual clutter. Categories joined: . 🌐 Best Use of Solana Canine AI Whisperer leverages the high throughput, ultra-low latency, and micro-transaction capabilities of the Solana blockchain to solve real-world problems in pet identity, health records, and behavioral training incentives. On-Chain Compressed NFT cNFT Passports: Mints immutable, tamper-proof canine digital identities encoding the pet's pedigree, age, weight, and microchip SHA-256 cryptographic hash. Ensures verifiable ownership and credentialing that shelters, veterinarians, and trainers can verify cryptographically across devices. Verifiable Canine Good Citizen CGC & Behavior Credentials: Issues on-chain verifiable credential badges e.g., Novice Obedience, AKC Canine Good Citizen, Therapy Dog Certified linked directly to cryptographic transaction signatures. TREATS Micro-Economy & Positive Reinforcement Engine: Implements tokenized behavioral quest verification e.g., 20-Min Quiet Crate Streak, Doorbell De-escalation Compliance, Loose-Leash Walking . Pet owners and trainers verify behavioral milestones, awarding TREATS tokens into their Solana wallet in real time. Integrated Solana Devnet Cluster & Faucet: Complete in-app cluster balance tracking, transaction signature verification, one-click airdrop requests, and clipboard signature tools. 6 Distinct Canine Inner Voice Personas: Tailored voice models matched to ethological archetypes: The Golden Goof Eager, high-energy, food-motivated The Bulldog Lord Deep, dignified, slightly grumpy The Shepherd Guardian Vigilant, crisp, tactical The Frenchie Sassy Fast-paced, demanding, playful The Husky Dramatic Theatrical, expressive, vocal The Whisperer Sage Calm, grounding, pack-leader tone Real-Time Translation Vocalization: When a dog's posture or acoustic bark is analyzed by Gemini, the resulting human-language translation is synthesized on the fly via ElevenLabs neural audio. Whisperer Coaching Spoken Responses: Users can listen to audible behavioral guidance and counter-conditioning instructions directly inside the interactive chat coach. Resilient Audio Fallback Architecture: Integrated multi-tier fallback that seamlessly bridges ElevenLabs neural streams with client-side Web Audio synthesis with dynamic pitch shifts deep guard tones to puppy pitch . Sub-Second Multimodal Visual Biometrics: Ingests real-time WebRTC camera streams and high-resolution photos to evaluate 6 subtle anatomical micro-markers simultaneously: Ear Base Tension flattened vs. pricked vs. neutral Commissure Lip Retraction loose pant vs. tight stress grimace Sclera Exposure Whale eye / fear indicators Spinal Rigidity & Weight Distribution forward territorial lean vs. rear avoidance load Pupil Dilation & Brow Furrowing Tail Carriage Angle and Oscillation Stiffness Structured Clinical Ethology Diagnostics: Gemini computes an exact Arousal Score , Cortisol Risk Tier Low, Moderate, Elevated, Critical , and immediate step-by-step de-escalation protocols. Acoustic Vocalization Intent Decoding: Correlates sound frequency harmonics Hz and decibel peaks with situational context to diagnose emotional drivers separation distress, predatory arousal, barrier frustration . Cesar Millan Pack Leadership Chat Coach: Multi-turn veterinary chat assistant grounded in ethological principles Exercise, Discipline, Affection in strict order for desensitization, crate transitions, and reactive leash behavior.