# I replaced the chat window for my local AI agent with a face

> Source: <https://dev.to/ghostvessel/i-replaced-the-chat-window-for-my-local-ai-agent-with-a-face-3e1k>
> Published: 2026-07-08 21:41:56+00:00

I run a local LLM agent (Hermes) on my own machine. The problem was never the model — it was the *interface*. I had a Telegram tab open all day just to talk to it: type a command, wait, read a wall of text back, scroll. It felt like texting a very capable stranger.

So I built ** Ghost Vessel** — a monitor-resident, video-call-style avatar that fronts the agent. The name is the whole idea: the

Here's what actually turned out to be interesting to build.

The core idea is an **output contract**. Instead of treating the agent's reply as text to print, I split every reply into three planes:

Emotion beats are inline tags the model emits in-band with its answer:

`[working]`

— the avatar puts on glasses and takes notes while a task runs`[confirm] deploy to prod?`

— pops a human-in-the-loop approve/cancel, and the agent blocks on your keypress`[happy]`

/ `[concerned]`

/ … — fine-grained facial expressionsSo "run the build, and if it passes, deploy" becomes a little *performance*: it looks busy while working, shows you the log as a card, then leans in and asks before the irreversible step. The text you'd have skim-read becomes something you glance at.

The obvious way to animate a face is live inference. I didn't want that — the GPU is busy running the actual model.

Instead the avatar is **~30 pre-rendered clips**, and the emotion beats just select and blend between them (blink-aligned seamless idle loops, a head-pose "settle gate" so an expression only reveals when the head is frontal). The avatar's runtime cost is basically video playback. Your GPU stays 100% on your LLM.

The tradeoff: no real-time lip-sync. I decided a believable *talking mouth loop* + expressive face reads as "a person on a call" far more than perfect phoneme-matching does, and it costs nothing at runtime. That single decision collapsed most of the hard engineering.

The agent integration is the part I was most unsure would work. It turned out clean: the app registers as a **connector**. The agent's gateway dials *out* to a local WebSocket the app hosts, and exchanges frames — so from the agent's side, the avatar is just "another channel," indistinguishable from Telegram.

Adapters for Hermes and OpenClaw are included, plus a demo responder so it runs with zero setup. And the chat pane serves the agent's **live slash-command menu** — type `/`

and you get the same 52 commands you'd see in the messenger, passed straight through.

The engine is **MIT** and usable on its own: clone it, drop a folder of clips named by emotion (`happy.mp4`

, `working.mp4`

, `idle.mp4`

, …), point it at your agent. Avatars are **pure-data bundles** — no code runs when you install one — so you can build your own; the full reproducible method is in the repo.

The part I'd most like feedback on is the **emotion-beat output format** — the contract that turns LLM text into a UI performance. Has anyone else built output contracts to drive an interface from model output? What broke, and what did models reliably get right vs. wrong? I'd genuinely like to compare notes.