I started thinking over why doesn't fireworks support voice models. There are really good opensource models available now, like parakeet, kokoro, Qwen ASR etc but no way to use it without managing a bunch of GPUs yourself.
Even LLMs like Gemma 4 used by voice agents are not supported.
Then I figured that the inference platform needs to be optimized differently for the kind of usecase you are using. Lets take an example for LLMs, not even STT and TTS. - Coding agents -> lot of cached input, needs to optimize for KV cache - Creation slides/blogs -> lots of output, needs to optimize for speculative decoding - Voice LLMs -> Cached input small output, not yet figured out on how to optimize this.
So TTS and STT is a completely different ballgame.
What I don't know is the timing, do people want to use open source models like kokoro, parakeet, Qwen etc RIGHT NOW?
Comments URL: [https://news.ycombinator.com/item?id=49186754](https://news.ycombinator.com/item?id=49186754)
Points: 2