{"slug": "live2d-body-for-hermes-agent", "title": "Live2D Body for Hermes Agent", "summary": "A developer released a transparent Live2D desktop pet app for macOS that integrates with Hermes AI agents to play lip-synced voice recordings through a local HTTP API, enabling health checks and cron reminders with a cute animated character.", "body_md": "A transparent Live2D desktop pet (Tauri 2 + pixi-live2d-display) with voice lip sync,\nexpressions, mouse tracking, and a local HTTP API. Currently ships the **Sparkle (火花)**\nmodel. Point your [hermes agent](https://github.com/NousResearch/hermes-agent) at it\nand every voice recording it generates is played on the pet — lip-synced, with a speech bubble.\n\n[Releases](https://github.com/Soundpulse/hermes-live2d/releases) ·\n[API reference](/Soundpulse/hermes-live2d/blob/main/docs/API.md) ·\n[Hermes skill](/Soundpulse/hermes-live2d/blob/main/skills/hermes-live2d-voice/SKILL.md) ·\n[Claude Code skill](/Soundpulse/hermes-live2d/blob/main/examples/claude-code-skill.md)\n\n**Useful for:**\n\n- Health checks\n- Cron reminders\n- Looking cute\n\nThis is the main use case. Your hermes agent runs somewhere (e.g. an EC2 box, chatting over\nthe Discord gateway); the pet runs on your Mac. Whenever the agent generates a TTS recording\nwith its `text_to_speech`\n\ntool, a small skill uploads that audio to the pet, which plays it with\nlip sync + expression + a speech bubble. Normal chat replies are **not** spoken — only recordings\nthe agent was already producing get routed.\n\n```\nhermes agent (remote)                         your Mac\n────────────────────                          ────────\ntext_to_speech ──► hermes-live2d-voice skill ──► POST /speak ──► Sparkle pet\n                     (speak.sh, base64)          (over Tailscale)   lip sync + bubble\n```\n\non both the Mac and the agent host (same tailnet) — this is how the remote agent reaches the pet without exposing it publicly.[Tailscale](https://tailscale.com)**A hermes agent wired to a TTS engine**— any engine works (ElevenLabs, etc.), as long as the agent produces audio files via its`text_to_speech`\n\ntool. That's the audio the pet plays.**The**— bundled in this repo at`hermes-live2d-voice`\n\nskill(`skills/hermes-live2d-voice/`\n\n`SKILL.md`\n\n+`speak.sh`\n\n); you install it on the agent host in step 3.\n\nRequirements: **macOS on Apple Silicon** (the app uses macOS-private window APIs and builds only\nfor `aarch64-apple-darwin`\n\n).\n\nGrab a build from [Releases](https://github.com/Soundpulse/hermes-live2d/releases), or build from\nsource:\n\n```\npnpm install\npnpm tauri build          # release .app / .dmg under src-tauri/target/release/bundle/\n# or, during development:\npnpm tauri dev\n```\n\n**Install the Live2D model.** The model assets are **not** included in this repo (they're\ngitignored) and are not redistributed here — download them and drop them in yourself:\n\n- Download the free\n**Spark (火花)** model from[booth.pm/en/items/8265367](https://booth.pm/en/items/8265367)and unzip it. - Place the unzipped model folder so its entry file is named exactly\n(rename the`Sparkle.model3.json`\n\n`.model3.json`\n\nif the download uses a different name), in one of:— runtime override, no rebuild needed. Loaded on the next launch and takes priority. Result:`~/.atri/model/`\n\n`~/.atri/model/Sparkle.model3.json`\n\n(plus its`.moc3`\n\n, textures, physics, expressions, etc. alongside).— bundled into the app at build time (use this when building from source; required, since the repo ships no model). Result:`public/model/`\n\n`public/model/Sparkle.model3.json`\n\n.\n\nTo use a different Live2D character entirely, put its files in `~/.atri/model/`\n\nwith the entry\nfile renamed to `Sparkle.model3.json`\n\n— the filename is what the app looks for. Note its\nexpressions/keyforms will differ (see the skill's model-swap note).\n\nLaunch the app. On first run it creates `~/.atri/config.json`\n\nand starts a local HTTP API on\n`http://127.0.0.1:3210`\n\n. Confirm it's up:\n\n```\ncurl http://127.0.0.1:3210/status\n# {\"ok\":true,\"message\":\"ATRI Live2D API is running\"}\n```\n\nThe API binds to loopback only (`127.0.0.1`\n\n) and has no auth, so it must **not** be exposed\ndirectly. Put it on your tailnet instead. On the Mac:\n\n```\ntailscale serve --bg 3210\n```\n\nThis publishes the pet at `https://<mac-name>.<tailnet>.ts.net`\n\n, reachable only by your own\ndevices.\n\nCopy the bundled skill into your hermes agent's skills directory:\n\n```\ncp -r skills/hermes-live2d-voice /path/to/hermes/skills/\n```\n\nIt's a standard `SKILL.md`\n\n+ `speak.sh`\n\nbundle. Tell the agent's environment where the pet lives:\n\n```\nexport ATRI_SPEAK_URL=\"https://<mac-name>.<tailnet>.ts.net\"\n```\n\nVerify from the agent host:\n\n```\ncurl -s \"$ATRI_SPEAK_URL/status\"\n# {\"ok\":true,\"message\":\"ATRI Live2D API is running\"}\n```\n\nThat's it. See [ skills/hermes-live2d-voice/SKILL.md](/Soundpulse/hermes-live2d/blob/main/skills/hermes-live2d-voice/SKILL.md)\nfor the exact contract, the expression list, and the graceful-degrade behavior when the Mac is\nasleep or offline.\n\n**Transparent desktop pet**— frameless, transparent, always-on-top; draggable and resizable** Mouse tracking**— eyes and head follow the cursor in real time, across multiple monitors** Voice lip sync**— mouth animation is driven by audio volume during playback** Expressions & motions**— 12 expressions (`exp1`\n\n–`exp12`\n\n) plus custom motion playback**Speech bubbles**— typewriter-style bubbles with configurable duration, anchored to the head** HTTP API**— local REST API (default port 3210) for external programs to drive the pet** Window memory**— remembers window position and size across launches** System tray**— tray icon to lock/unlock and quit\n\nOnce the app is running, the API listens on `http://127.0.0.1:3210`\n\n.\n\n```\n# Health check\ncurl http://127.0.0.1:3210/status\n\n# Make the pet speak (text + audio + expression)\ncurl -X POST http://127.0.0.1:3210/speak \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"text\": \"hello!\", \"audio_url\": \"file:///path/to/audio.wav\", \"expression\": 1}'\n\n# Switch expression (id 1–12, or name exp1–exp12)\ncurl -X POST http://127.0.0.1:3210/expression \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"name\": \"exp1\"}'\n\n# Show a speech bubble\ncurl -X POST http://127.0.0.1:3210/bubble \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"text\": \"thinking...\", \"duration\": 3000}'\n```\n\n`/speak`\n\nalso accepts inline audio as base64 (`audio_data`\n\n+ `audio_format`\n\n) instead of an\n`audio_url`\n\n— that's how the remote hermes skill sends recordings across the network. Full\nendpoint reference: [docs/API.md](/Soundpulse/hermes-live2d/blob/main/docs/API.md).\n\nThe model exposes **12 expressions**, addressed as `expression: 1`\n\n–`12`\n\n(names `exp1`\n\n–`exp12`\n\n, ids\nare 1-based); omit `expression`\n\nfor the neutral/default face. Each id maps to a specific mood on\nthe loaded model — see the mood table in\n[ skills/hermes-live2d-voice/SKILL.md](/Soundpulse/hermes-live2d/blob/main/skills/hermes-live2d-voice/SKILL.md#expression-guide) for\nthe current mapping. Query the live id/name list any time with\n\n`curl http://127.0.0.1:3210/expressions`\n\n.Config lives at `~/.atri/config.json`\n\n, created on first launch:\n\n```\n{\n  \"api_port\": 3210\n}\n```\n\n- Window position/size are saved to\n`~/.atri/window_state.json`\n\n. - Uploaded\n`/speak`\n\naudio is written to`~/.atri/tmp/`\n\nand overwritten on each call. - Drop a custom Live2D model under\n`~/.atri/model/`\n\nto load it instead of the bundled Sparkle model on startup.\n\nThe bundled character is the free **Spark (火花)** Live2D model by 夜墨ww\n([booth.pm/en/items/8265367](https://booth.pm/en/items/8265367)) — Sparkle from Honkai: Star Rail.\n\n**Claude Code**—is a TTS-agnostic example skill: generate speech with your own TTS (GPT-SoVITS, Edge-TTS, …) and drive the pet after each reply.`examples/claude-code-skill.md`\n\n**GPT-SoVITS voice (optional)**— for a Japanese ATRI voice, a GPT-SoVITS model is hosted at[VoidShine/atri-sovits](https://huggingface.co/VoidShine/atri-sovits). Not needed for the hermes path, where the agent supplies its own TTS audio.\n\n| Layer | Technology |\n|---|---|\n| Desktop framework |\n|\n\n[PixiJS 6](https://pixijs.com)+[pixi-live2d-display](https://github.com/guansss/pixi-live2d-display)[Axum](https://github.com/tokio-rs/axum)[GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS)**macOS only**— relies on macOS-private APIs (transparent window, click-through, etc.); no Windows or Linux support.** Apple Silicon only**— release target is`aarch64-apple-darwin`\n\n.**Model-bound parameters**— expression/keyform mappings are hardcoded for the bundled model; swapping in a different Live2D model requires code changes.**Volume-driven lip sync**— mouth movement maps from audio volume, not viseme-accurate phonemes.** Local, unauthenticated API**— the HTTP API listens on`127.0.0.1`\n\nwith no auth; only expose it to trusted devices (e.g. a private tailnet), never the public internet.**Mouse tracking via Tauri**—`cursor_position()`\n\nbehavior can vary across macOS versions.\n\nProject code is released under the MIT license.\n\nUse of the Live2D Cubism SDK and the character model assets is subject to their respective\nlicenses. The bundled **Spark (火花)** model is by 夜墨ww\n([booth.pm/en/items/8265367](https://booth.pm/en/items/8265367)); follow its terms.", "url": "https://wpnews.pro/news/live2d-body-for-hermes-agent", "canonical_source": "https://github.com/Soundpulse/hermes-live2d", "published_at": "2026-07-13 02:46:58+00:00", "updated_at": "2026-07-13 03:05:33.839453+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools"], "entities": ["NousResearch", "Soundpulse", "Hermes Agent", "Sparkle", "Tailscale", "ElevenLabs", "ATRI Live2D API"], "alternates": {"html": "https://wpnews.pro/news/live2d-body-for-hermes-agent", "markdown": "https://wpnews.pro/news/live2d-body-for-hermes-agent.md", "text": "https://wpnews.pro/news/live2d-body-for-hermes-agent.txt", "jsonld": "https://wpnews.pro/news/live2d-body-for-hermes-agent.jsonld"}}