{"slug": "i-created-a-24-7-ai-avatar-that-streams-without-human-intervention-only-eyes-and", "title": "I Created a 24/7 AI Avatar That Streams Without Human Intervention — Only 'Verification,' 'Eyes,' and 'Ears' Remain for Humans", "summary": "A developer built a system where a 3D avatar live-streams autonomously, with humans only registering the show. The system automatically creates the broadcast, spins up a cloud GPU pod, renders the avatar, responds to viewer comments and tips, and ends the stream, with the scheduler managing the lifecycle to prevent runaway costs. The developer found that the more unattended the operation, the clearer the roles that must remain with humans become.", "body_md": "📝 Originally published (in Japanese) at\n\n[forge.workstyle.tech].\n\n番組の時間になる → Let me hand you the finished piece.\n\nI built a system where a 3D avatar live-streams entirely on its own. The only thing a human does is **register the show**. On the day itself, nobody touches anything.\n\n```\nShowtime arrives\n  → the broadcast is created automatically (YouTube)\n  → a cloud GPU Pod spins up\n  → the avatar renders in a page and starts pushing over RTMP\n  → the broadcast transitions to live\n  → it responds to viewer comments with speech\n  → it reacts to tips based on the amount\n  → when comments dry up, it raises its own topics from the show's theme\n  → when the runtime is up, it gives a closing greeting\n  → the stream ends, the Pod is destroyed, the archive remains\n```\n\nI worked through this system with an AI agent over several weeks — design, implementation, deployment, fault injection, and long-running verification. The first half of this post covers the overall architecture and **why I split it the way I did**. The second half covers what became visible through the process: **the roles that were left to humans and only humans**. The further you push toward unattended operation, the more sharply the parts you can never hand off come into focus.\n\n```\n┌────────────────────────┐\n│   Show Management UI   │  register a show (character / theme / runtime / destinations)\n└──────────┬─────────────┘\n           │\n┌──────────▼─────────────┐   create / health-check / destroy Pods\n│       Scheduler        │───────────────────────────┐\n│  (stream lifecycle)    │                           │  create / go live / end broadcasts\n└──────────┬─────────────┘                           │\n           │ run (one stream execution)              │\n┌──────────▼─────────────┐               ┌───────────▼────────────┐\n│    Dialogue Server     │◀── Redis ─────│     Chat Collector     │\n│ (utterance generation) │    Stream     │      (YT / Twitch)     │\n└──────────┬─────────────┘               └────────────────────────┘\n           │ WebSocket (display only)\n┌──────────▼─────────────┐\n│     Renderer Pod       │  headless Chromium + ffmpeg\n│         (GPU)          │──▶ tee ──▶ YouTube / Twitch\n└────────────────────────┘\n```\n\nThis was the very first decision. **A stream begins, continues, and ends** — it has a lifespan. And if it doesn't end, the billing doesn't either.\n\nIf you bury that lifespan management inside the dialogue logic, a bug in the dialogue turns directly into runaway costs. By splitting it out, the scheduler only has to watch one thing: is the stream alive, has it ended, is the Pod gone?\n\nThat's the scheduler's entire set of responsibilities:\n\nWithout those last two, you end up with GPU charges piling up while no video ever appears.\n\nYouTube comments come from polling the Data API. Twitch comes over IRC. Tips and subs come from EventSub. **Every source works differently.**\n\nIf you wire all of that straight into the dialogue server, the dialogue logic gets dirtier every time you add a platform. So I put a dedicated collection component in front and **normalized everything into a common internal format** before pushing it onto a Redis Stream.\n\n```\nYouTube chat    ┐\nTwitch IRC      ├→ normalize → Redis Stream → Dialogue Server\nTwitch EventSub ┘\n```\n\nFrom the dialogue server's point of view, there's exactly one input: a stream of events. No platform-specific knowledge has to live there.\n\nModeration (banned words, throttling repeat spam from the same user) also lives on the collection side. **Dropping the dirty stuff at the entrance** keeps every layer behind it simpler.\n\nConversation during a stream behaves nothing like web request handling. **You can't say two things at once.**\n\nEven when several comments arrive simultaneously, the avatar has one mouth. So speech is fully serialized, with a priority queue deciding the order.\n\nWhen comments flood in, there's also logic to batch several of them into a single utterance. Answering one at a time makes it look, from the viewer's side, like the avatar is still stuck on a comment from ages ago.\n\nRenderers go down. GPU hosts get flaky, browsers crash, ffmpeg dies.\n\nSo I designed it so that **the stream's state (what has been said, who it talked to) never lives in the page**. State sits in a Redis snapshot, and the page is a **display-only client** that connects and receives the current state.\n\nThat means if the page dies, reloading and reconnecting picks up right where it left off. It also avoids the dumb failure mode of **re-doing the opening greeting on every reconnect** — the snapshot already says \"greeting done.\"\n\nThe renderer runs on a different Pod, a different cloud, a different GPU. Its whole job is: open a page, capture the screen and audio, push it to RTMP.\n\nThat separation gives you:\n\n**Put the fragile parts where breaking doesn't hurt.**\n\nFor reference, a few numbers measured during verification.\n\n| Metric | Value |\n|---|---|\n| Pod creation → live transition | 95 s |\n| Concurrent streams per GPU | 4 avatars (720p30) |\n| Cost per avatar | ¥7,600/month (24h) / ¥2,500/month (8h per day) |\n| Comment response latency | 25–45 s (15–30 s of which is platform viewer delay) |\n| Automatic recovery from failure | 41 s (full recovery from a server restart) |\n| Continuous operation test | 2 hours (no memory growth, all 53 turns measured) |\n\nThe recovery numbers came from actually deleting Pods and restarting servers. Verification included **actually breaking things and confirming they heal.**\n\nThe technically hardest part turned out to be neither the dialogue nor the rendering — it was **ending reliably**.\n\nFailures to start are obvious immediately, because nobody can watch. Failures to *end* announce themselves via the invoice, or via \"wait, it's still streaming\" the next morning. **The scary thing about automation isn't failure — it's success that keeps going.** That's what hit home hardest.\n\nI ended up giving every component a condition along the lines of \"if I decide I'm in a bad state, I stop.\"\n\nThat's the architecture. From here I want to talk about how the work itself went. I did all of this alongside an AI agent, and what I noticed partway through was that **the moments a human got called in fell into exactly three categories**. Nearly everything else closed out on the agent's side.\n\nFirst, the work no human touched:\n\nThe measured numbers above, including the 41-second recovery, came out of that process. **The range of what actually gets done is far wider than you'd expect.**\n\nThis was by far the largest bucket.\n\nWhat these share is that they all demand **proof that you are you**. This isn't a matter of technical difficulty — it's a domain where delegation isn't supposed to be possible. If an agent *could* do these on your behalf, that service's identity verification would be broken.\n\nThe practically important part is that **these create waiting**. Twenty-four hours from phone verification to activation isn't something code can shorten.\n\nDuring development, I **kept a running homework list for the human**. Separate \"what the agent can move on right now\" from \"what can't proceed until a human does it,\" and get the homework done first. Skip that and you'll finish the implementation only to sit through a 24-hour wait. And in fact, I punted on homework a few times and lost a full day to \"waiting on authorization.\"\n\nThe second bucket: **things you can only judge by looking and listening.**\n\n| Judgment | Why a machine can't settle it |\n|---|---|\n| Setting lip-sync delay to 0.10 s | \"Looks in sync\" is a perceptual question. There's no correct number |\n| Whether render flicker is acceptable | It shows up in neither fps, nor errors, nor GPU utilization |\n| Whether a voice sounds natural | Waveform metrics don't line up with subjective impressions |\n| The overall \"watchable quality\" of a stream | A holistic call |\n\nThe most telling case was render flicker. **Every performance metric stayed normal while the character's face strobed.** fps, errors, GPU utilization — not one of them indicated anything wrong. Changing a graphics setting fixed it, but the only reason anyone *noticed* it was broken is that a human watched the video.\n\nThat changed how I ran things: **any change touching rendering or audio has to be confirmed by a human seeing and hearing the real thing before it's finalized.** This is less a limitation of AI capability than a property of the problem — **the criterion for the judgment exists only inside human perception**. I wrote earlier that \"the renderer is disposable,\" but the step that finally signs off on its quality stayed with human eyes and ears.\n\nThe third bucket: decisions about going public.\n\nTechnically, all of these can be executed at any time. Whether they *should* be is a separate question. **Judgments where accountability sits with a human** stay with the human — as a matter of authority, not capability.\n\nI drew the line from the start: verify with unlisted streams, and switch to public only on a human's call. The agent operates on the assumption of that line too.\n\nTo summarize, three things stayed with the human:\n\nPut the other way around: **everything else runs on the agent's side.** Design, implementation, and deployment, plus fault injection, measurement, and root-cause analysis.\n\nThe thing that paid off most in practice was **identifying this boundary at the very start of the project**. Hand the \"human homework\" over early and there's no wait left when the implementation lands.\n\nThe other thing I noticed is that all three share a property: **you could do them on someone's behalf, but you shouldn't.** These aren't technical limits. That's exactly why I don't expect the boundary to move much as capabilities improve.\n\nOn architecture:\n\nOn process:\n\nUnattended streaming lets you hand off almost every step to the machine. GPU selection, browser media APIs, the quirks of platform APIs, fault-tolerant design — each one has enough traps for its own article, and all of it got handled on the agent's side. What was left standing at the end was: being who you say you are, judging with your own eyes and ears, and carrying the responsibility of publishing. My conclusion is that **rather than hunting for \"what AI can't do,\" it's practically faster to decide up front what the human should do.**", "url": "https://wpnews.pro/news/i-created-a-24-7-ai-avatar-that-streams-without-human-intervention-only-eyes-and", "canonical_source": "https://dev.to/orca_forge/i-created-a-247-ai-avatar-that-streams-without-human-intervention-only-verification-eyes-67d", "published_at": "2026-08-27 05:46:25+00:00", "updated_at": "2026-08-27 06:18:10.356189+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["YouTube", "Twitch", "Redis", "Chromium", "ffmpeg"], "alternates": {"html": "https://wpnews.pro/news/i-created-a-24-7-ai-avatar-that-streams-without-human-intervention-only-eyes-and", "markdown": "https://wpnews.pro/news/i-created-a-24-7-ai-avatar-that-streams-without-human-intervention-only-eyes-and.md", "text": "https://wpnews.pro/news/i-created-a-24-7-ai-avatar-that-streams-without-human-intervention-only-eyes-and.txt", "jsonld": "https://wpnews.pro/news/i-created-a-24-7-ai-avatar-that-streams-without-human-intervention-only-eyes-and.jsonld"}}