Is there an response length limit for the inference API?

Hugging Face's Inference API returns only 2-3 sentences per response regardless of the model used, according to a user report. The issue may be resolved by setting the `max_new_tokens` parameter higher, ensuring the model supports long-form generation, and checking the context window limits.

Hi, I am testing the Inference API with different models to rewrite texts. But no matter which model I choose, only about 2 - 3 sentences are returned as response. How can this be adjusted? A few things to check: - Make sure you’re setting max new tokens or the equivalent generation parameter high enough. The default value can be quite small. - Verify that the model you’re using supports long-form text generation. - Check whether your input plus output exceeds the model’s context window. - If you’re using the Inference API, inspect the full response to see if it indicates the generation stopped because it reached a token limit finish reason or similar . If you can share the model name and your request payload, it will be easier to identify the cause.