I am using the inference provider with the model moonshotai/Kimi-K2.6:fireworks-ai
, but I am experiencing frequent Gateway Timeout errors when using the Hugging Face Inference Provider.
This issue seems to occur specifically when the context is long and reasoning is set to a high level, where it is expected that the model will take significantly more time to generate a response.
The Fireworks AI backend itself appears to be working fine, as I do not encounter this issue when using the Fireworks AI API directly.
Could someone help me understand what might be causing this or suggest a possible solution?
@michellehbn Can u pls help with this? Because direct Fireworks calls succeed, I’d first isolate the HF router’s timeout budget from the provider’s own generation timeout. For long-context/high-reasoning requests, record time-to-first-token, total generation time, and whether the 504 happens before any stream data; then retry only before partial output and use a bounded fallback rather than resending the whole context blindly. Full disclosure: I’m building Your Model, a multi-model OpenAI-compatible API. If you need another hosted endpoint to test against, I can provide some test credits.