Building a Streaming Chatbot with Node.js

Most backend tutorials show synchronous calls: the API generates the full response and only then prints it, after several seconds of waiting. That’s not how real chat products work — in interfaces like ChatGPT or Claude, the response starts appearing word by word, before it’s even complete. That’s called streaming, and it’s what we’re going to build today.

What you’ll learn

Requirements

Step 1: Project setup

mkdir llm-chatbot-tutorial
cd llm-chatbot-tutorial
npm init -y
...

Install the dependencies. We use the openai package because Groq’s API is compatible with its format — the same SDK, just pointed at a different URL:

npm install openai dotenv

Step 2: Protect your credentials from the start

Create .gitignore :

node_modules/
.env

Create .env.example (this one does get committed):

GROQ_API_KEY=your_api_key_here

And your real .env (this one is never committed):

GROQ_API_KEY=your_api_key_here

Step 3: The chatbot code

Create chatbot.js :

require("dotenv").config();
const OpenAI = require("openai");

const client = new OpenAI({
  apiKey: process.env.GROQ_API_KEY,
  baseURL: "https://api.groq.com/openai/v1",
});

// Check console.groq.com/docs/models for the current model list —
// providers rotate models periodically.
const MODEL = "openai/gpt-oss-120b";

async function chat(userMessage) {
  console.log(`\n📝 User: ${userMessage}\n`);
  console.log("🤖 Assistant: ");

  try {
    const stream = await client.chat.completions.create({
      model: MODEL,
      stream: true,
      messages: [
        { role: "user", content: userMessage },
      ],
    });

    for await (const chunk of stream) {
      const text = chunk.choices[0]?.delta?.content || "";
      process.stdout.write(text);
    }

    console.log("\n");
  } catch (error) {
    console.error("Error:", error.message);
  }
}

async function main() {
  console.log("=== Streaming Chatbot with Node.js ===\n");
  await chat("What is the capital of Argentina?");
  await chat("Explain what Node.js is in two sentences.");
  await chat("Give me 3 tips for learning to code.");
  console.log("\n✅ Done!");
}

main();

What stream: true actually does

Without streaming, the API waits until the full response is generated and only then returns it as a single block. With stream: true , the server instead sends the response as a sequence of small fragments ( delta objects) as the model generates each token.

The for await loop processes each fragment as soon as it arrives. We use

process.stdout.write(text) instead of console.log(text) on purpose: console.log adds a line break after every call, which would break the live-typing visual effect.

This pattern stays the same across providers — only the SDK import and a couple of field names change. Once you understand this shape, you can swap providers with minimal effort.

Step 4: Run it

node chatbot.js

You should see the three questions answered one after another, with text streaming in real time —not appearing all at once.

**If you hit an error: **- Check that .env is in the project root - Confirm the variable is named exactly GROQ_API_KEY - Verify the configured model is still available at console.groq.com/docs/models

Step 5: Customize it

Change the questions — edit the chat(...) calls inside main() .

Add a system prompt to control the assistant’s tone or role:

messages: [
  { role: "system", content: "You are a concise, friendly coding tutor." },
  { role: "user", content: userMessage },
],

Swap the model — just change the MODEL constant. Always check the provider’s current model list before publishing or deploying, since free-tier models get deprecated and replaced over time.

Conclusion

You now have a working, streaming LLM chatbot in under 60 lines of Node.js — and, more importantly, you understand the streaming pattern that powers every modern AI chat interface. This is the foundation for anything more advanced: a web-based chat UI, a Slack bot, or a backend service that talks to an LLM.

Next in this series: containerizing this app with Docker and deploying it to the cloud.