02:49
2026-10-16
discuss.huggingface.co
ai-infrastructure
Two models on one 8 GB GPU: making the LLM wait for the TTS after every sentence stopped my voice assistant from stuttering
Running a large language model and a text-to-speech model simultaneously on one 8 GB GPU delayed the first PCM audio to 2.3–2.5 s after the TTS request, versus 1.0–1.2 s when the two models take turns…