Live2D Body for Hermes Agent

A developer released a transparent Live2D desktop pet app for macOS that integrates with Hermes AI agents to play lip-synced voice recordings through a local HTTP API, enabling health checks and cron reminders with a cute animated character.

A transparent Live2D desktop pet Tauri 2 + pixi-live2d-display with voice lip sync, expressions, mouse tracking, and a local HTTP API. Currently ships the Sparkle 火花 model. Point your hermes agent https://github.com/NousResearch/hermes-agent at it and every voice recording it generates is played on the pet — lip-synced, with a speech bubble. Releases https://github.com/Soundpulse/hermes-live2d/releases · API reference /Soundpulse/hermes-live2d/blob/main/docs/API.md · Hermes skill /Soundpulse/hermes-live2d/blob/main/skills/hermes-live2d-voice/SKILL.md · Claude Code skill /Soundpulse/hermes-live2d/blob/main/examples/claude-code-skill.md Useful for: - Health checks - Cron reminders - Looking cute This is the main use case. Your hermes agent runs somewhere e.g. an EC2 box, chatting over the Discord gateway ; the pet runs on your Mac. Whenever the agent generates a TTS recording with its text to speech tool, a small skill uploads that audio to the pet, which plays it with lip sync + expression + a speech bubble. Normal chat replies are not spoken — only recordings the agent was already producing get routed. hermes agent remote your Mac ──────────────────── ──────── text to speech ──► hermes-live2d-voice skill ──► POST /speak ──► Sparkle pet speak.sh, base64 over Tailscale lip sync + bubble on both the Mac and the agent host same tailnet — this is how the remote agent reaches the pet without exposing it publicly. Tailscale https://tailscale.com A hermes agent wired to a TTS engine — any engine works ElevenLabs, etc. , as long as the agent produces audio files via its text to speech tool. That's the audio the pet plays. The — bundled in this repo at hermes-live2d-voice skill skills/hermes-live2d-voice/ SKILL.md + speak.sh ; you install it on the agent host in step 3. Requirements: macOS on Apple Silicon the app uses macOS-private window APIs and builds only for aarch64-apple-darwin . Grab a build from Releases https://github.com/Soundpulse/hermes-live2d/releases , or build from source: pnpm install pnpm tauri build release .app / .dmg under src-tauri/target/release/bundle/ or, during development: pnpm tauri dev Install the Live2D model. The model assets are not included in this repo they're gitignored and are not redistributed here — download them and drop them in yourself: - Download the free Spark 火花 model from booth.pm/en/items/8265367 https://booth.pm/en/items/8265367 and unzip it. - Place the unzipped model folder so its entry file is named exactly rename the Sparkle.model3.json .model3.json if the download uses a different name , in one of:— runtime override, no rebuild needed. Loaded on the next launch and takes priority. Result: ~/.atri/model/ ~/.atri/model/Sparkle.model3.json plus its .moc3 , textures, physics, expressions, etc. alongside .— bundled into the app at build time use this when building from source; required, since the repo ships no model . Result: public/model/ public/model/Sparkle.model3.json . To use a different Live2D character entirely, put its files in ~/.atri/model/ with the entry file renamed to Sparkle.model3.json — the filename is what the app looks for. Note its expressions/keyforms will differ see the skill's model-swap note . Launch the app. On first run it creates ~/.atri/config.json and starts a local HTTP API on http://127.0.0.1:3210 . Confirm it's up: curl http://127.0.0.1:3210/status {"ok":true,"message":"ATRI Live2D API is running"} The API binds to loopback only 127.0.0.1 and has no auth, so it must not be exposed directly. Put it on your tailnet instead. On the Mac: tailscale serve --bg 3210 This publishes the pet at https://<mac-name .<tailnet .ts.net , reachable only by your own devices. Copy the bundled skill into your hermes agent's skills directory: cp -r skills/hermes-live2d-voice /path/to/hermes/skills/ It's a standard SKILL.md + speak.sh bundle. Tell the agent's environment where the pet lives: export ATRI SPEAK URL="https://<mac-name .<tailnet .ts.net" Verify from the agent host: curl -s "$ATRI SPEAK URL/status" {"ok":true,"message":"ATRI Live2D API is running"} That's it. See skills/hermes-live2d-voice/SKILL.md /Soundpulse/hermes-live2d/blob/main/skills/hermes-live2d-voice/SKILL.md for the exact contract, the expression list, and the graceful-degrade behavior when the Mac is asleep or offline. Transparent desktop pet — frameless, transparent, always-on-top; draggable and resizable Mouse tracking — eyes and head follow the cursor in real time, across multiple monitors Voice lip sync — mouth animation is driven by audio volume during playback Expressions & motions — 12 expressions exp1 – exp12 plus custom motion playback Speech bubbles — typewriter-style bubbles with configurable duration, anchored to the head HTTP API — local REST API default port 3210 for external programs to drive the pet Window memory — remembers window position and size across launches System tray — tray icon to lock/unlock and quit Once the app is running, the API listens on http://127.0.0.1:3210 . Health check curl http://127.0.0.1:3210/status Make the pet speak text + audio + expression curl -X POST http://127.0.0.1:3210/speak \ -H 'Content-Type: application/json' \ -d '{"text": "hello ", "audio url": "file:///path/to/audio.wav", "expression": 1}' Switch expression id 1–12, or name exp1–exp12 curl -X POST http://127.0.0.1:3210/expression \ -H 'Content-Type: application/json' \ -d '{"name": "exp1"}' Show a speech bubble curl -X POST http://127.0.0.1:3210/bubble \ -H 'Content-Type: application/json' \ -d '{"text": "thinking...", "duration": 3000}' /speak also accepts inline audio as base64 audio data + audio format instead of an audio url — that's how the remote hermes skill sends recordings across the network. Full endpoint reference: docs/API.md /Soundpulse/hermes-live2d/blob/main/docs/API.md . The model exposes 12 expressions , addressed as expression: 1 – 12 names exp1 – exp12 , ids are 1-based ; omit expression for the neutral/default face. Each id maps to a specific mood on the loaded model — see the mood table in skills/hermes-live2d-voice/SKILL.md /Soundpulse/hermes-live2d/blob/main/skills/hermes-live2d-voice/SKILL.md expression-guide for the current mapping. Query the live id/name list any time with curl http://127.0.0.1:3210/expressions .Config lives at ~/.atri/config.json , created on first launch: { "api port": 3210 } - Window position/size are saved to ~/.atri/window state.json . - Uploaded /speak audio is written to ~/.atri/tmp/ and overwritten on each call. - Drop a custom Live2D model under ~/.atri/model/ to load it instead of the bundled Sparkle model on startup. The bundled character is the free Spark 火花 Live2D model by 夜墨ww booth.pm/en/items/8265367 https://booth.pm/en/items/8265367 — Sparkle from Honkai: Star Rail. Claude Code —is a TTS-agnostic example skill: generate speech with your own TTS GPT-SoVITS, Edge-TTS, … and drive the pet after each reply. examples/claude-code-skill.md GPT-SoVITS voice optional — for a Japanese ATRI voice, a GPT-SoVITS model is hosted at VoidShine/atri-sovits https://huggingface.co/VoidShine/atri-sovits . Not needed for the hermes path, where the agent supplies its own TTS audio. | Layer | Technology | |---|---| | Desktop framework | | PixiJS 6 https://pixijs.com + pixi-live2d-display https://github.com/guansss/pixi-live2d-display Axum https://github.com/tokio-rs/axum GPT-SoVITS https://github.com/RVC-Boss/GPT-SoVITS macOS only — relies on macOS-private APIs transparent window, click-through, etc. ; no Windows or Linux support. Apple Silicon only — release target is aarch64-apple-darwin . Model-bound parameters — expression/keyform mappings are hardcoded for the bundled model; swapping in a different Live2D model requires code changes. Volume-driven lip sync — mouth movement maps from audio volume, not viseme-accurate phonemes. Local, unauthenticated API — the HTTP API listens on 127.0.0.1 with no auth; only expose it to trusted devices e.g. a private tailnet , never the public internet. Mouse tracking via Tauri — cursor position behavior can vary across macOS versions. Project code is released under the MIT license. Use of the Live2D Cubism SDK and the character model assets is subject to their respective licenses. The bundled Spark 火花 model is by 夜墨ww booth.pm/en/items/8265367 https://booth.pm/en/items/8265367 ; follow its terms.