Live2D Body for Hermes Agent

A transparent Live2D desktop pet (Tauri 2 + pixi-live2d-display) with voice lip sync, expressions, mouse tracking, and a local HTTP API. Currently ships the Sparkle (火花) model. Point your hermes agent at it and every voice recording it generates is played on the pet — lip-synced, with a speech bubble.

Releases · API reference · Hermes skill · Claude Code skill

Useful for:

• Health checks
• Cron reminders
• Looking cute

This is the main use case. Your hermes agent runs somewhere (e.g. an EC2 box, chatting over the Discord gateway); the pet runs on your Mac. Whenever the agent generates a TTS recording with its text_to_speech

tool, a small skill uploads that audio to the pet, which plays it with lip sync + expression + a speech bubble. Normal chat replies are not spoken — only recordings the agent was already producing get routed.

hermes agent (remote)                         your Mac
────────────────────                          ────────
text_to_speech ──► hermes-live2d-voice skill ──► POST /speak ──► Sparkle pet
                     (speak.sh, base64)          (over Tailscale)   lip sync + bubble

on both the Mac and the agent host (same tailnet) — this is how the remote agent reaches the pet without exposing it publicly.TailscaleA hermes agent wired to a TTS engine— any engine works (ElevenLabs, etc.), as long as the agent produces audio files via itstext_to_speech

tool. That's the audio the pet plays.The— bundled in this repo athermes-live2d-voice

skill(skills/hermes-live2d-voice/

SKILL.md

+speak.sh

); you install it on the agent host in step 3.

Requirements: macOS on Apple Silicon (the app uses macOS-private window APIs and builds only for aarch64-apple-darwin

).

Grab a build from Releases, or build from source:

pnpm install
pnpm tauri build          # release .app / .dmg under src-tauri/target/release/bundle/
pnpm tauri dev

Install the Live2D model. The model assets are not included in this repo (they're gitignored) and are not redistributed here — download them and drop them in yourself:

• Download the free Spark (火花) model frombooth.pm/en/items/8265367and unzip it. - Place the unzipped model folder so its entry file is named exactly (rename theSparkle.model3.json

.model3.json

if the download uses a different name), in one of:— runtime override, no rebuild needed. Loaded on the next launch and takes priority. Result:~/.atri/model/

~/.atri/model/Sparkle.model3.json

(plus its.moc3

, textures, physics, expressions, etc. alongside).— bundled into the app at build time (use this when building from source; required, since the repo ships no model). Result:public/model/

public/model/Sparkle.model3.json

.

To use a different Live2D character entirely, put its files in ~/.atri/model/

with the entry file renamed to Sparkle.model3.json

— the filename is what the app looks for. Note its expressions/keyforms will differ (see the skill's model-swap note).

Launch the app. On first run it creates ~/.atri/config.json

and starts a local HTTP API on http://127.0.0.1:3210

. Confirm it's up:

curl http://127.0.0.1:3210/status

The API binds to loopback only (127.0.0.1

) and has no auth, so it must not be exposed directly. Put it on your tailnet instead. On the Mac:

tailscale serve --bg 3210

This publishes the pet at https://<mac-name>.<tailnet>.ts.net

, reachable only by your own devices.

Copy the bundled skill into your hermes agent's skills directory:

cp -r skills/hermes-live2d-voice /path/to/hermes/skills/

It's a standard SKILL.md

• speak.sh

bundle. Tell the agent's environment where the pet lives:

export ATRI_SPEAK_URL="https://<mac-name>.<tailnet>.ts.net"

Verify from the agent host:

curl -s "$ATRI_SPEAK_URL/status"

That's it. See skills/hermes-live2d-voice/SKILL.md for the exact contract, the expression list, and the graceful-degrade behavior when the Mac is asleep or offline.

Transparent desktop pet— frameless, transparent, always-on-top; draggable and resizable** Mouse tracking**— eyes and head follow the cursor in real time, across multiple monitors** Voice lip sync**— mouth animation is driven by audio volume during playback** Expressions & motions**— 12 expressions (exp1

–exp12

) plus custom motion playbackSpeech bubbles— typewriter-style bubbles with configurable duration, anchored to the head** HTTP API**— local REST API (default port 3210) for external programs to drive the pet** Window memory**— remembers window position and size across launches** System tray**— tray icon to lock/unlock and quit

Once the app is running, the API listens on http://127.0.0.1:3210

.

curl http://127.0.0.1:3210/status

curl -X POST http://127.0.0.1:3210/speak \
  -H 'Content-Type: application/json' \
  -d '{"text": "hello!", "audio_url": "file:///path/to/audio.wav", "expression": 1}'

curl -X POST http://127.0.0.1:3210/expression \
  -H 'Content-Type: application/json' \
  -d '{"name": "exp1"}'

curl -X POST http://127.0.0.1:3210/bubble \
  -H 'Content-Type: application/json' \
  -d '{"text": "thinking...", "duration": 3000}'

/speak

also accepts inline audio as base64 (audio_data

• audio_format

) instead of an audio_url

— that's how the remote hermes skill sends recordings across the network. Full endpoint reference: docs/API.md.

The model exposes 12 expressions, addressed as expression: 1

–12

(names exp1

–exp12

, ids are 1-based); omit expression

for the neutral/default face. Each id maps to a specific mood on the loaded model — see the mood table in skills/hermes-live2d-voice/SKILL.md for the current mapping. Query the live id/name list any time with

curl http://127.0.0.1:3210/expressions

.Config lives at ~/.atri/config.json

, created on first launch:

{
  "api_port": 3210
}

• Window position/size are saved to ~/.atri/window_state.json

. - Uploaded /speak

audio is written to~/.atri/tmp/

and overwritten on each call. - Drop a custom Live2D model under ~/.atri/model/

to load it instead of the bundled Sparkle model on startup.

The bundled character is the free Spark (火花) Live2D model by 夜墨ww (booth.pm/en/items/8265367) — Sparkle from Honkai: Star Rail.

Claude Code—is a TTS-agnostic example skill: generate speech with your own TTS (GPT-SoVITS, Edge-TTS, …) and drive the pet after each reply.examples/claude-code-skill.md

GPT-SoVITS voice (optional)— for a Japanese ATRI voice, a GPT-SoVITS model is hosted atVoidShine/atri-sovits. Not needed for the hermes path, where the agent supplies its own TTS audio.

Layer	Technology
Desktop framework

PixiJS 6+pixi-live2d-displayAxumGPT-SoVITSmacOS only— relies on macOS-private APIs (transparent window, click-through, etc.); no Windows or Linux support.** Apple Silicon only**— release target isaarch64-apple-darwin

.Model-bound parameters— expression/keyform mappings are hardcoded for the bundled model; swapping in a different Live2D model requires code changes.Volume-driven lip sync— mouth movement maps from audio volume, not viseme-accurate phonemes.** Local, unauthenticated API**— the HTTP API listens on127.0.0.1

with no auth; only expose it to trusted devices (e.g. a private tailnet), never the public internet.Mouse tracking via Tauri—cursor_position()

behavior can vary across macOS versions.

Project code is released under the MIT license.

Use of the Live2D Cubism SDK and the character model assets is subject to their respective licenses. The bundled Spark (火花) model is by 夜墨ww (booth.pm/en/items/8265367); follow its terms.

source & further reading

github.com — original article