Why Fireworks doesn't support Voice AI

Fireworks, an AI inference platform, does not support voice AI models such as Parakeet, Kokoro, and Qwen ASR because voice workloads require different optimization strategies than text LLMs, according to a Hacker News post. The post notes that while open-source voice models are available, using them requires managing GPUs, and the platform has not yet figured out how to optimize for voice LLMs, which have small cached input and output. The author questions whether users want to use these open-source models immediately.

I started thinking over why doesn't fireworks support voice models. There are really good opensource models available now, like parakeet, kokoro, Qwen ASR etc but no way to use it without managing a bunch of GPUs yourself. Even LLMs like Gemma 4 used by voice agents are not supported. Then I figured that the inference platform needs to be optimized differently for the kind of usecase you are using. Lets take an example for LLMs, not even STT and TTS. - Coding agents - lot of cached input, needs to optimize for KV cache - Creation slides/blogs - lots of output, needs to optimize for speculative decoding - Voice LLMs - Cached input small output, not yet figured out on how to optimize this. So TTS and STT is a completely different ballgame. What I don't know is the timing, do people want to use open source models like kokoro, parakeet, Qwen etc RIGHT NOW? Comments URL: https://news.ycombinator.com/item?id=49186754 https://news.ycombinator.com/item?id=49186754 Points: 2 Comments: 0