{"slug": "stop-waiting-for-the-full-ai-response-stream-tokens-in-python", "title": "Stop Waiting for the Full AI Response: Stream Tokens in Python", "summary": "A developer demonstrates how to stream tokens from OpenAI-compatible APIs in Python, showing that adding stream=True to a chat completion request returns chunks that can be printed as they arrive, improving perceived responsiveness. The pattern was tested with an OpenAI-compatible endpoint through APIHubRelay.", "body_md": "Most AI applications wait for the model to generate the complete answer before showing anything to the user.\n\nFor short answers, that may be acceptable. For longer responses, it can make the application feel slow—even when the model is already generating tokens.\n\nStreaming solves this by displaying each part of the response as soon as it arrives.\n\nA standard OpenAI-compatible request may look like this:\n\n``` python\nimport os\nfrom openai import OpenAI\n\nclient = OpenAI(\n    api_key=os.environ[\"AI_API_KEY\"],\n    base_url=os.environ[\"AI_BASE_URL\"],\n)\n\nresponse = client.chat.completions.create(\n    model=os.environ[\"AI_MODEL\"],\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"Explain API gateways in three sentences.\",\n        }\n    ],\n)\n\nprint(response.choices[0].message.content)\n```\n\nThis works, but nothing is printed until the complete response has arrived.\n\nEnable streaming by adding `stream=True`\n\n:\n\n```\nstream = client.chat.completions.create(\n    model=os.environ[\"AI_MODEL\"],\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"Explain API gateways in three sentences.\",\n        }\n    ],\n    stream=True,\n)\n```\n\nThe request now returns a sequence of chunks instead of one completed response.\n\nLoop through those chunks and print the available content:\n\n```\nfor chunk in stream:\n    content = chunk.choices[0].delta.content\n\n    if content:\n        print(content, end=\"\", flush=True)\n\nprint()\n```\n\nThe user can now see the answer while it is being generated.\n\n``` python\nimport os\nfrom openai import OpenAI\n\nclient = OpenAI(\n    api_key=os.environ[\"AI_API_KEY\"],\n    base_url=os.environ[\"AI_BASE_URL\"],\n)\n\nstream = client.chat.completions.create(\n    model=os.environ[\"AI_MODEL\"],\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"Explain API gateways in three sentences.\",\n        }\n    ],\n    stream=True,\n)\n\nfor chunk in stream:\n    content = chunk.choices[0].delta.content\n\n    if content:\n        print(content, end=\"\", flush=True)\n\nprint()\n```\n\nKeeping the API key, base URL, and model name in environment variables also makes it easier to change providers without rewriting the application logic.\n\nStreaming is especially helpful for:\n\nRemember that model capabilities and streaming formats can vary between providers. Verify support for your selected model and handle empty chunks, connection failures, and interrupted streams before using this pattern in production.\n\nI tested this pattern with an OpenAI-compatible endpoint through [APIHubRelay](https://apihubrelay.com/).\n\nWhat should the next example cover: streaming in Node.js, error handling, or automatic retries?", "url": "https://wpnews.pro/news/stop-waiting-for-the-full-ai-response-stream-tokens-in-python", "canonical_source": "https://dev.to/chen_qin/stop-waiting-for-the-full-ai-response-stream-tokens-in-python-110o", "published_at": "2026-08-03 06:38:11+00:00", "updated_at": "2026-08-03 07:13:35.724324+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models"], "entities": ["OpenAI", "APIHubRelay"], "alternates": {"html": "https://wpnews.pro/news/stop-waiting-for-the-full-ai-response-stream-tokens-in-python", "markdown": "https://wpnews.pro/news/stop-waiting-for-the-full-ai-response-stream-tokens-in-python.md", "text": "https://wpnews.pro/news/stop-waiting-for-the-full-ai-response-stream-tokens-in-python.txt", "jsonld": "https://wpnews.pro/news/stop-waiting-for-the-full-ai-response-stream-tokens-in-python.jsonld"}}