Prevent Hallucinated Bot Commands: Build an LLM Tool Loop for Telegram in Node.js

A developer outlined a Node.js pattern for Telegram bots that replaces regex and JSON.parse extraction of LLM output with native function calling, defining explicit JSON schemas for actions like get_order and create_booking. In the described approach, the model returns a structured intent that the application executes locally, keeping API keys and database access out of the prompt, with results fed back to the model for a final reply. The writeup argues the naive prompt-only approach fails predictably when models wrap JSON in Markdown fences or fabricate order IDs.

When building a Telegram bot powered by a Large Language Model LLM , developers often start by feeding the user's message directly to an inference API and piping the raw string output back to bot.sendMessage . This works fine for generic Q&A, but breaks down completely when the user asks the bot to execute a concrete action, such as "Check the status of order 9482" or "Book a table for 2 people at 7 PM". If you rely on prompt engineering alone—asking the LLM to return plain text or raw JSON formatted like {"action": "check order", "id": 9482} —the system will inevitably fail in production. The model will hallucinate missing parameters, output conversational text around the JSON block, format output with unparseable Markdown fences, or generate fabricated database records when an order ID is missing. Even worse, attempting to resolve system tasks by feeding sensitive system context or credentials into the LLM system prompt risks exposing your Telegram bot token or internal database structure. To build a reliable transactional bot, you must implement strict function calling tool use in Node.js. In this pattern, the LLM does not execute actions directly and never sees your API keys. Instead, the model outputs a structured intent JSON, your Node.js application executes the local code such as querying PostgreSQL or calling a REST service , and the results are fed back into the model to generate a final human-readable response for the Telegram user. A common anti-pattern in early AI Telegram bots is attempting to parse intended actions out of freeform assistant responses using regular expressions or JSON.parse . Consider a typical customer support bot built with Node.js and a standard Telegram library like grammY or node-telegram-bot-api . The naive implementation tries to extract parameters directly from unstructured model text. // BAD: Relying on unstructured text and regex extraction bot.on 'message:text', async ctx = { const userText = ctx.message.text; const prompt = You are a support bot. If the user wants order status, reply ONLY with JSON: {"action": "get order", "id": 123}. User: ${userText} ; const response = await openai.chat.completions.create { model: 'gpt-4o-mini', messages: { role: 'user', content: prompt } , } ; const replyText = response.choices 0 .message.content; try { // This will fail when the model returns {% endraw %} json ... {% raw %} or adds text like "Sure, here is..." const command = JSON.parse replyText ; if command.action === 'get order' { const order = await db.orders.find command.id ; await ctx.reply Order status: ${order.status} ; return; } } catch err { // Falls back to sending raw text, which often leaks system instructions or fails completely await ctx.reply replyText ; } } ; This approach breaks under predictable operational conditions: json ... JSON.parse throws an unhandled syntax error. 1234 or 1 to satisfy the requested JSON structure, causing your bot to query incorrect database records. To fix this, you must separate model intent from execution logic using explicit API schemas and a controlled tool loop. Native function calling requires defining explicit JSON schemas for every utility function your Telegram bot supports. These schemas inform the LLM about available tools, required parameter types, and property descriptions. Create a dedicated definitions file tools.js that defines two concrete actions: fetching an order status get order and creating a table booking create booking . js // tools.js export const botTools = { type: 'function', function: { name: 'get order', description: 'Fetch the status and delivery details of an existing customer order.', parameters: { type: 'object', properties: { order id: { type: 'string', description: 'The unique alphanumeric order identifier, e.g., ORD-5502.', }, }, required: 'order id' , }, }, }, { type: 'function', function: { name: 'create booking', description: 'Reserve a table for a customer at the restaurant.', parameters: { type: 'object', properties: { guest name: { type: 'string', description: 'Full name of the primary guest making the reservation.', }, party size: { type: 'integer', description: 'Number of guests attending 1 to 10 .', }, booking time: { type: 'string', description: 'ISO 8601 formatted date and time string for the reservation.', }, }, required: 'guest name', 'party size', 'booking time' , }, }, }, ; By specifying required fields in the schema, the inference engine knows it must ask the user for clarification if essential values like order id or party size are missing from the input message, rather than guessing them. Never pass database connection strings, internal server tokens, or your Telegram Bot Token into tool descriptions or system messages. The model only receives function arguments; actual execution takes place within your Node.js runtime. The core architecture of a reliable Telegram AI agent consists of a multi-turn conversation loop within a single HTTP request handler: tool role. Here is the full implementation using Node.js, grammY , and the official @openai/api SDK. python // bot.js import { Bot } from 'grammY'; import OpenAI from 'openai'; import { botTools } from './tools.js'; const bot = new Bot process.env.TELEGRAM BOT TOKEN ; const openai = new OpenAI { apiKey: process.env.OPENAI API KEY } ; // Local mock database handlers async function getOrderDetails orderId { const mockDb = { 'ORD-101': { status: 'Shipped', carrier: 'FedEx', tracking: 'FX-99281' }, 'ORD-102': { status: 'Processing', carrier: 'Pending', tracking: 'N/A' }, }; return mockDb orderId || { error: 'Order not found in database.' }; } async function createTableBooking guestName, partySize, bookingTime { // Validate and write to database const bookingId = BK-${Math.floor 1000 + Math.random 9000 } ; return { success: true, booking id: bookingId, details: { guestName, partySize, bookingTime }, }; } bot.on 'message:text', async ctx = { const userMessage = ctx.message.text; // Initialize short-term memory array for this request lifecycle const messages = { role: 'system', content: 'You are an automated Telegram customer support agent. Answer questions concisely using provided tools. Do not invent order statuses or fake reservations.', }, { role: 'user', content: userMessage }, ; try { // Initial call to LLM with tool definitions let response = await openai.chat.completions.create { model: 'gpt-4o-mini', messages: messages, tools: botTools, tool choice: 'auto', } ; let responseMessage = response.choices 0 .message; // Process tool calls if requested by the model if responseMessage.tool calls && responseMessage.tool calls.length 0 { // Append the assistant's request for tool execution to message history messages.push responseMessage ; for const toolCall of responseMessage.tool calls { const functionName = toolCall.function.name; const functionArgs = JSON.parse toolCall.function.arguments ; let functionResult; if functionName === 'get order' { functionResult = await getOrderDetails functionArgs.order id ; } else if functionName === 'create booking' { functionResult = await createTableBooking functionArgs.guest name, functionArgs.party size, functionArgs.booking time ; } else { functionResult = { error: 'Unknown function call.' }; } // Return tool output to the model conversation history messages.push { tool call id: toolCall.id, role: 'tool', name: functionName, content: JSON.stringify functionResult , } ; } // Second call to LLM to process tool results into natural response response = await openai.chat.completions.create { model: 'gpt-4o-mini', messages: messages, } ; responseMessage = response.choices 0 .message; } // Send final synthesized text to Telegram user if responseMessage.content { await ctx.reply responseMessage.content ; } } catch error { console.error 'Execution Error:', error ; await ctx.reply 'Sorry, I encountered an issue processing your request. Please try again later.' ; } } ; bot.start ; When moving an LLM-powered Telegram bot to production, relying on basic functionality will leave you open to race conditions, billing spikes, and hanging Telegram updates. Adhere to these key practices: An LLM can occasionally loop infinitely by requesting tools back-to-back if a tool returns an error payload. Always wrap your execution loop in a strict iteration counter e.g., maximum 3 tool turns per user update . If the limit is reached, terminate execution gracefully and inform the user. Telegram messages hard-cap at 4,096 characters per message payload. When passing database results back to the LLM, truncate large data arrays on your Node.js backend before inserting them into the tool output message. If the final model response exceeds 4,000 characters, split the message before invoking ctx.reply . Telegram Webhooks expect an HTTP 200 OK response within 10 to 60 seconds depending on bot architecture . Running multiple LLM inference calls and external database queries in sequence can easily exceed webhook execution budgets. For long-running operations: await ctx.replyWithChatAction 'typing' . bot.api.sendMessage chatId, text once complete. Treat arguments produced by JSON.parse toolCall.function.arguments as untrusted user input. Validate argument types and sanitize strings before feeding them directly into SQL parameters, external shell tools, or secondary API integrations. Building transactional Telegram integrations requires strict tool boundary separation to guarantee stability under heavy production traffic. If you need assistance scaling complex custom workflows or mini-app backends, get in touch with BotCreator https://botservice.biz — studio that ships Telegram bots / Mini Apps.