Building a Streaming Chatbot with Node.js

A developer published a tutorial showing how to build a streaming LLM chatbot in under 60 lines of Node.js using the OpenAI SDK pointed at Groq's API endpoint. The guide demonstrates the stream: true pattern, in which the server sends response fragments as delta objects token by token, and uses process.stdout.write to render text live rather than waiting for the full completion. It also covers credential handling via .env files, adding a system prompt, and swapping models or providers with minimal code changes.

Most backend tutorials show synchronous calls: the API generates the full response and only then prints it, after several seconds of waiting. That’s not how real chat products work — in interfaces like ChatGPT or Claude, the response starts appearing word by word, before it’s even complete. That’s called streaming , and it’s what we’re going to build today. What you’ll learn Requirements Step 1: Project setup mkdir llm-chatbot-tutorial cd llm-chatbot-tutorial npm init -y ... Install the dependencies. We use the openai package because Groq’s API is compatible with its format — the same SDK, just pointed at a different URL: npm install openai dotenv Step 2: Protect your credentials from the start Create .gitignore : node modules/ .env Create .env.example this one does get committed : GROQ API KEY=your api key here And your real .env this one is never committed : GROQ API KEY=your api key here Step 3: The chatbot code Create chatbot.js : js require "dotenv" .config ; const OpenAI = require "openai" ; const client = new OpenAI { apiKey: process.env.GROQ API KEY, baseURL: "https://api.groq.com/openai/v1", } ; // Check console.groq.com/docs/models for the current model list — // providers rotate models periodically. const MODEL = "openai/gpt-oss-120b"; async function chat userMessage { console.log \n📝 User: ${userMessage}\n ; console.log "🤖 Assistant: " ; try { const stream = await client.chat.completions.create { model: MODEL, stream: true, messages: { role: "user", content: userMessage }, , } ; for await const chunk of stream { const text = chunk.choices 0 ?.delta?.content || ""; process.stdout.write text ; } console.log "\n" ; } catch error { console.error "Error:", error.message ; } } async function main { console.log "=== Streaming Chatbot with Node.js ===\n" ; await chat "What is the capital of Argentina?" ; await chat "Explain what Node.js is in two sentences." ; await chat "Give me 3 tips for learning to code." ; console.log "\n✅ Done " ; } main ; What stream: true actually does Without streaming, the API waits until the full response is generated and only then returns it as a single block. With stream: true , the server instead sends the response as a sequence of small fragments delta objects as the model generates each token. The for await loop processes each fragment as soon as it arrives. We use process.stdout.write text instead of console.log text on purpose: console.log adds a line break after every call, which would break the live-typing visual effect. This pattern stays the same across providers — only the SDK import and a couple of field names change. Once you understand this shape, you can swap providers with minimal effort. Step 4: Run it node chatbot.js You should see the three questions answered one after another, with text streaming in real time —not appearing all at once. If you hit an error: - Check that .env is in the project root - Confirm the variable is named exactly GROQ API KEY - Verify the configured model is still available at console.groq.com/docs/models Step 5: Customize it Change the questions — edit the chat ... calls inside main . Add a system prompt to control the assistant’s tone or role: messages: { role: "system", content: "You are a concise, friendly coding tutor." }, { role: "user", content: userMessage }, , Swap the model — just change the MODEL constant. Always check the provider’s current model list before publishing or deploying, since free-tier models get deprecated and replaced over time. Conclusion You now have a working, streaming LLM chatbot in under 60 lines of Node.js — and, more importantly, you understand the streaming pattern that powers every modern AI chat interface. This is the foundation for anything more advanced: a web-based chat UI, a Slack bot, or a backend service that talks to an LLM. Next in this series: containerizing this app with Docker and deploying it to the cloud.