How Many AI Avatars Can One GPU Handle? Real-World Test Reveals 4 Avatars at ¥7,600 Each per Month

A developer's real-world test on an RTX 4000 Ada GPU revealed that four 3D AI avatars can stream simultaneously at 720p30, with GPU usage at only 26% and the CPU as the bottleneck. The cost per avatar is approximately ¥7,600 per month for 24/7 streaming, and the project identified that using NVENC could increase capacity.

📝 Originally published in Japanese at forge.workstyle.tech . We're developing an unmanned system where 3D avatars automatically handle live streaming. The system boots up a cloud GPU pod at the scheduled start time, the renderer assembles and streams the video, and then the pod is discarded when the segment ends. Since there's no human oversight, three factors directly impact the success of the business and service quality: "how many avatars can run simultaneously," "how the system recovers from failures," and "how quickly it starts up." These questions couldn't be answered through estimates alone. Renting a GPU for a few hours costs only a few hundred yen. In this article, we'll share three stories of how we measured and designed the system, following the structure of "stumbling block → cause → solution." These three aspects seem independent but are actually interconnected. Faster startup enabled practical host switching, and understanding capacity allowed us to set prices. Let's dive into each one. The first number we desperately needed was "how many avatars can run on a single GPU." Without this, we couldn't determine pricing, and without pricing, we couldn't assess the business viability. Estimates were useless, so we measured it. Here are the results: | Item | Measured Value | |---|---| | GPU | RTX 4000 Ada Community type, $0.28/hour | | Simultaneous Streams | 4 avatars maintaining 720p30 in real-time Recorded segment: 89 seconds / 89 seconds | | GPU Usage | 26% | | Bottleneck | CPU 16 vCPU side saturated first | | Estimated Upper Limit | 5–6 avatars | And the pricing: | Operation Mode | Monthly Cost per Avatar | |---|---| | 24/7 Streaming | Approximately ¥7,600 | | 8-hour Daily Schedule | Approximately ¥2,500 | A common mistake in measuring simultaneous execution is focusing solely on FPS. For streaming, this is insufficient. You need to check if the recorded segment length matches the actual time . The reason is simple: when rendering fails, the pipeline doesn't "stutter" but skips time . We experienced a case where a 90-second animation recorded only 6 seconds. The FPS logs looked fine, but the output was truncated. So, we set the success criteria as: Run N avatars simultaneously for 89 seconds, All output files must have an actual length of 89 seconds. With 4 avatars, all files were 89 seconds / 89 seconds. This confirmed that "4 avatars can run simultaneously." We also verified that the screen capture rate was 33fps. Running 4 avatars resulted in 26% GPU usage . This means the GPU had more than triple the capacity. The bottleneck was the CPU 16 vCPU . The breakdown explains why: | Process | Uses | |---|---| | 3D Scene Rendering | GPU | | Frame Extraction | CPU / Transfer | | H.264 Encoding | CPU if software encoding | | Audio Mixing and Muxing | CPU | | RTMP Streaming | CPU / Network | The GPU only handles rendering, while the rest of the streaming pipeline relies on the CPU. Assuming "we're renting a GPU" leads to focusing on GPU specs, but the actual limiting factor was the number of vCPUs. This observation suggests another improvement: using a hardware encoder NVENC would free up CPU resources , potentially increasing the number of avatars. When choosing a GPU, "NVENC availability" should be a criterion. When packing multiple avatars into one host, we made one implementation change. The renderer originally sent audio from the page to ffmpeg via a named pipe fifo . If this path is shared across processes, host sharing fails. The second avatar would grab the same pipe, causing audio interference. We solved this by making the path unique per port number. /tmp/audio.fifo → Not shareable /tmp/audio-<port .fifo → Unique per avatar Shared resources like temporary files, fixed ports, lock files, and cache directories become issues when sharing hosts. Identifying these beforehand makes measurements smoother. The calculation is straightforward: $0.28/hour × 720 hours/month = $201.6/month per GPU $201.6 ÷ 4 avatars = $50.4/avatar ≈ ¥7,600/avatar at ¥150/USD Operational hours significantly impact costs. At 24/7 operation, it's ¥7,600 per avatar, but our system schedules streams and automatically terminates pods afterward. With 8 hours daily, it's ¥2,500 per avatar —a threefold difference. Running streams during off-peak hours when no viewers are present is simply wasting money. The key learning was treating this as a scheduling problem "which time slots to use?" rather than a necessity for constant operation. Besides GPU hourly rates, consider these: Even with known capacity, a single GPU might not be stable. Using affordable community-type GPUs where individuals or businesses rent out excess GPU capacity , we encountered this issue: | Run | Host | Behavior | |---|---|---| | run3 | Host A | No disconnections for 10 minutes | | run4 | Host B | Crashed every 60–150 seconds → recovered → crashed again | The image, settings, and code were identical. Only the physical host differed. Initially, we suspected our code and checked recovery logs, but eventually concluded it wasn't our fault. The solution was to switch hosts automatically. In live streaming, this issue isn't resolved by simple restarts. Each renderer recovery: Viewers see a stream where the avatar reintroduces itself every minute. The more robust the recovery, the stranger the symptoms , making it a tricky problem. We implemented a two-component solution. One alone wasn't enough. Layer 1: Renderer Self-Reports Failure Time-Window Burst Detection The renderer already had self-recovery for crashes e.g., ffmpeg or page crashes . We added a time window to recovery counts: If recoveries exceed 4 within a 600-second window, Terminate the process with exit code 1. The key is not using cumulative counts. In long streams, even healthy hosts recover a few times nightly. Cumulative counters would flag healthy long streams. We focus on "concentrated failures in a short time" using a sliding window. Self-termination seems counterintuitive but is the most reliable way to signal "this instance is faulty" to higher layers. Layer 2: Scheduler Monitors Pod Health and Switches Hosts The scheduler, managing stream lifecycles, now monitors pod status: EXITED including Layer 1 self-termination or Treating 404s as "disappearance" rather than errors was crucial. Treating them as exceptions would trigger retry loops. Deleted pods also return 404s on deletion requests, which is harmless the pod is already gone . While logs show ERROR, it can be ignored. After implementation, we tested by manually deleting a pod during a stream: Pod deletion → Detected disappearance in 32 seconds → Recreated pod on a different host → Renderer started, stream resumed → Program ended automatically, pod discarded Without failure injection, recovery code often remains untested. Crude failure methods are fine ; manually triggering failures is reliable. Here, deleting the pod via API mimicked a faulty host. Honestly, it's not perfect. Switching hosts mid-stream can cause the stream to end prematurely. YouTube automatically terminates streams on disconnection, sometimes closing before the new pod starts. Since we don't recreate stream keys, the program ends. The solution is restarting faster than the stream closes . This ties into the next section. Host switching is only practical with fast startup. We reduced startup time to 95 seconds by baking everything into the renderer image. Some cases still fail. Community-type GPUs are cheap $0.24–0.28/hour but inconsistent. We use them as follows: | Use Case | Choice | |---|---| | Development experiments, short tests | Community type cost-effective | | Production streams, long tests | Secure type operated by businesses, more consistent | Regardless of choice, implement host switching. Even secure types fail; frequency differs. Host switching effectiveness depends on startup speed. Initially, startup to streaming took ~4 minutes. This delay directly impacted program starts and caused silence during host switches. Using a custom image reduced this to 95 seconds . Here's what we changed. For quick development, we used this setup: apt Vulkan tools, Japanese fonts, ffmpeg, etc. npm install This worked during development. No image rebuilds were needed for code changes , speeding up iteration. During GPU experimentation, this agility was invaluable. Issues arose when moving to production. 1. Simply Slow Apt index updates, package downloads, and npm dependency resolution happen every time. A 4-minute delay is unacceptable for a streaming service. 2. Increased External Dependencies Each startup connects to apt mirrors, npm registries, and our asset server. If any service is slow, streaming delays. Our reliability depended on external services' performance. An incident occurred when our asset server's nginx, configured with worker processes auto , spawned 196 workers, hitting memory limits and preventing stream starts containers with auto use the host's core count . Components added for streaming ended up blocking it. 3. Occasional Write Failures at Startup This was tricky. Writing to the filesystem overlayfs immediately after container startup sometimes failed , affecting ICD settings. Failures meant GPU rendering wouldn't initialize, and the startup would continue with CPU rendering. Chromium falling back to CPU rendering shows no errors , resulting in "streams starting with choppy video." Like the "time skips" issue in Chapter 1, retries were added but didn't address the root cause. We created a custom image, moving all startup tasks to build time: FROM mcr.microsoft.com/playwright:v1.54.0-noble RUN apt-get update && apt-get install -y \ ffmpeg \ fonts-noto-cjk \ Japanese fonts prevents tofu text vulkan-tools mesa-utils \ xvfb \ && rm -rf /var/lib/apt/lists/ Bake NVIDIA EGL/Vulkan ICD settings prevents startup write failures COPY icd/ /usr/share/ COPY app/ /app/ RUN cd /app && npm ci --omit=dev Now, startup only involves pulling the image and starting the process. Pod creation to YouTube live transition takes 95 seconds —a 2.5x improvement from ~4 minutes. Honestly, baking has trade-offs. Initial image pulls are heavy. Playwright's official image is large, and adding more increases size. On uncached hosts, pulls can take minutes we observed 9 minutes even with the old method . Thus, 95 seconds assumes a cached host. First pulls take longer, which must be estimated honestly. We chose baking because time variance is reduced. Startup setups depend on multiple external services, causing unpredictability. Image pulls are a one-time, cacheable dependency. Consistent slowness is easier to manage than occasional slowness. apt implicitly uses the latest versions The last point is most significant. Startup setups leave reproducibility to chance. While it appeared as a speed problem, it was rooted in reproducibility and dependencies. We determined three key metrics through measurement: Capacity Reliability Startup Speed worker processes auto in containersThese aspects are interconnected. Faster startup enabled practical host switching, which allowed using affordable community GPUs for longer, and measuring on these GPUs provided pricing data. Cloud resources aren't uniform. Design for instance replacement, not repair —and measure capacity, startup time, and pricing with actual tests. It's the most cost-effective approach.