{"slug": "ai-human-in-the-loop-news-digest-triage-telegram-bot", "title": "AI Human-in-the-loop: News Digest Triage Telegram Bot", "summary": "A developer built a Telegram bot that uses AI to categorize tech news stories and lets users confirm or override the tags with a single tap, implementing a human-in-the-loop system to catch AI hallucinations. The bot fetches stories from Lobste.rs, suggests tags via OpenAI, and stores confirmed entries in a JSONL file. The project demonstrates how to add a control layer above AI models for more reliable decision-making.", "body_md": "# AI Human-in-the-loop: News Digest Triage Telegram Bot\n\n*Building agentic AI? I co-run a 6-week cohort where you ship a production-ready agent, not another API wrapper.*\n\nIn my [trend digest article](/blog/python-ai-trend-digest-asyncio-protocol/) I shared a quick tool to keep on top of tech trends, but it's a one-way street: the model gives information, but I still have to decide what to do with it. Let's build the second half: a Telegram bot that shows me each story, guesses a tag, and lets me confirm or overrule it with one tap.\n\n## Human-in-the-loop (HITL): the model proposes, you decide\n\nAI makes suggestions but it can hallucinate, so it's important to have a human in the loop to catch mistakes. The model does the work of categorizing, the human makes the final decision. This is a good example of [the control layer above the model](/blog/control-layer-is-the-product/) and it's where you can make AI more reliable.\n\nThis is what we teach in week 4 of our [Agentic AI cohort](https://pythonagenticai.com) where things come together: expense parsing, AI category suggestion, and the human in the loop to confirm it. This requires the bot to keep state, route responses, and a way to be wrong gracefully. Below is a smaller version so you can get a taste for how this works.\n\nWe'll build it in seven steps. Clone [the repo](https://github.com/bbelderbos/telegram-hitl) to follow along step by step.\n\n## Step 1: create the bot and get a token\n\nTelegram bots are created by another bot. Open Telegram and search for **@BotFather** (it has a blue checkmark):\n\n- Send\n`/newbot`\n\n. - Give it a display name (anything).\n- Give it a username ending in\n`bot`\n\nthat is globally unique, e.g.`alice_trend_bot`\n\n.\n\nBotFather replies with a token like `123456789:ABCdef...`\n\n. Treat it like a password. Put it in a `.env`\n\nfile next to your script (or `export`\n\nin your shell), together with your OpenAI key:\n\n```\nTELEGRAM_BOT_TOKEN=123456789:ABCdef...\nOPENAI_API_KEY=sk-...\n```\n\nIf the token ever leaks, send `/revoke`\n\nto BotFather for a fresh one.\n\n## Step 2: the dependencies\n\nThe whole thing is one file. I use a [PEP 723 header](/blog/python-ai-trend-digest-asyncio-protocol/) so `uv run`\n\nresolves everything into its own environment, no virtualenv to manage. Put this at the top of `trend_triage_bot.py`\n\n:\n\n```\n# /// script\n# requires-python = \">=3.12\"\n# dependencies = [\n#   \"python-telegram-bot>=21\",\n#   \"openai>=1.40\",\n#   \"httpx\",\n#   \"python-decouple\",\n#   \"pydantic\",\n# ]\n# ///\n```\n\nIf you would rather build this inside an existing project, the equivalent is:\n\n```\nuv init && uv add python-telegram-bot openai httpx python-decouple pydantic\n```\n\nThen the imports and a few constants:\n\n``` python\nimport json\nimport logging\nfrom pathlib import Path\nfrom typing import Literal, Protocol\n\nimport httpx\nfrom decouple import config\nfrom openai import AsyncOpenAI\nfrom pydantic import BaseModel\nfrom telegram import InlineKeyboardButton, InlineKeyboardMarkup, Message, Update\nfrom telegram.ext import (\n    Application,\n    CallbackQueryHandler,\n    CommandHandler,\n    ContextTypes,\n)\n\nlogger = logging.getLogger(__name__)\n\nTAGS = [\"read\", \"lib\", \"tool\", \"skip\"]\nDEFAULT_TOPIC = \"rust\"\nREADING_LIST = Path(\"reading_list.jsonl\")\nLOBSTERS_FEED = \"https://lobste.rs/t/{tag}.json\"\n```\n\n## Step 3: fetch the stories\n\n[Lobsters](https://lobste.rs) has a per-tag JSON feed, no auth required: `https://lobste.rs/t/rust.json`\n\nreturns the latest Rust-tagged stories, `.../t/python.json`\n\nthe Python ones, and so on. It's a tighter, more engineering-focused signal than a broad keyword search, and parameterizing the tag is what lets `/digest rust`\n\nand `/digest python`\n\nhit the same code. A `Story`\n\nis just a title and a URL.\n\nLet's set up the model and fetch the latest five stories for a given tag:\n\n``` python\nfrom pydantic import BaseModel, HttpUrl\n\nclass Story(BaseModel):\n    title: str\n    url: HttpUrl\n\nasync def fetch_stories(tag: str, *, limit: int = 5) -> list[Story]:\n    async with httpx.AsyncClient(\n        timeout=10, headers={\"User-Agent\": \"trend-triage-bot\"}\n    ) as http:\n        response = await http.get(LOBSTERS_FEED.format(tag=tag))\n        response.raise_for_status()\n        return [\n            Story.model_validate(\n                {\n                    \"title\": story[\"title\"],\n                    \"url\": story[\"url\"] or f\"https://lobste.rs/s/{story['short_id']}\",\n                }\n            )\n            for story in response.json()[:limit]\n            if story.get(\"title\")\n        ]\n```\n\nTwo small details: Lobsters expects a `User-Agent`\n\nheader, and a text/discussion post has an empty `url`\n\n, so we fall back to its comments page (`/s/{short_id}`\n\n), the same pattern you'd use for an HN self-post.\n\nThe `*`\n\nin the function signature makes the `limit`\n\nkeyword-only, so you have to call `fetch_stories(\"rust\", limit=10)`\n\n, which is a nice safeguard against accidentally changing the default.\n\n## Step 4: let the LLM propose a tag\n\nThe model picks one of `TAGS`\n\n. As the digest topic is variable (`/digest rust`\n\n, `/digest python`\n\n), the tags have to be topic-agnostic, so they describe what a story *is* (`read`\n\n/ `lib`\n\n/ `tool`\n\n), not anything Rust- or Python-specific.\n\nContent-type beats intent here: \"is this a tool or a library\" is answerable from a headline, whereas \"will I read this or build with it\" depends on me, not the title. And a tag the model can't infer is a tag you end up correcting every time.\n\nAnd using [structured outputs](/blog/structured-outputs-pydantic-openai/) I get typed values back, not strings I have to parse and second-guess; consistent data types are the foundation of reliable AI.\n\n```\nSYSTEM = (\n    \"Tag this software/tech headline with one of: \"\n    \"read (an article, post, or tutorial), \"\n    \"lib (a library, framework, or package you import), \"\n    \"tool (a CLI, app, or utility you run). \"\n    \"Use 'skip' only if it is off-topic or clickbait.\"\n)\n\nclass TagChoice(BaseModel):\n    tag: Literal[\"read\", \"lib\", \"tool\", \"skip\"]\n\nclass Classifier(Protocol):\n    async def tag(self, story: Story) -> str: ...\n\nclass OpenAIClassifier:\n    def __init__(self, api_key: str, model: str = \"gpt-4o-mini\") -> None:\n        self._client = AsyncOpenAI(api_key=api_key)\n        self._model = model\n\n    async def tag(self, story: Story) -> str:\n        completion = await self._client.beta.chat.completions.parse(\n            model=self._model,\n            messages=[\n                {\"role\": \"system\", \"content\": SYSTEM},\n                {\"role\": \"user\", \"content\": story.title},\n            ],\n            response_format=TagChoice,\n        )\n        choice = completion.choices[0].message.parsed\n        return choice.tag if choice else \"skip\"\n\ndef _build_classifier() -> Classifier:\n    return OpenAIClassifier(config(\"OPENAI_API_KEY\"))\n```\n\n`_build_classifier()`\n\nconstructs the client at runtime, not at import; we call it once in `main()`\n\nand stash the result (more on that in step 7).\n\nThis decoupling allows a test to inject a fake tagger without touching OpenAI. It's the same lazy-wiring trick I used [demonstrating the repository pattern](/blog/repository-pattern-swappable-data-sources/). The `Protocol`\n\nmeans any class with an `async tag()`\n\nmethod drops in. Protocols are more flexible here, because they don't require inheritance like ABCs do, so the test double doesn't have to know about the real classifier at all.\n\nFiling a tagged story is a one-liner to a JSONL file. JSONL (or JSON Lines) is a way to store structured data; each line contains a single, valid JSON object.\n\n``` php\ndef save_to_reading_list(story: Story, tag: str) -> None:\n    with READING_LIST.open(\"a\") as f:\n        f.write(json.dumps({\"tag\": tag, **story.model_dump(mode=\"json\")}) + \"\\n\")\n```\n\nNote that `model_dump()`\n\nhands back a pydantic `Url`\n\nobject (`HttpUrl`\n\n) that `json.dumps`\n\ncan't serialize; `mode=\"json\"`\n\ncoerces it to a string first.\n\n## Step 5: the keyboard that highlights the guess\n\nThe AI's pick is prefixed with `>>`\n\n, but every other tag is one tap away. The `callback_data`\n\nstays plain (`tag:read`\n\n) so the handler never has to strip the decoration:\n\n``` php\ndef triage_keyboard(suggested: str) -> InlineKeyboardMarkup:\n    buttons = [\n        InlineKeyboardButton(\n            f\">> {tag}\" if tag == suggested else tag,\n            callback_data=f\"tag:{tag}\",\n        )\n        for tag in TAGS\n    ]\n    rows = [buttons[i : i + 3] for i in range(0, len(buttons), 3)]\n    return InlineKeyboardMarkup(rows)\n```\n\nThe marker goes in *front*, and that detail matters: Telegram clips long button labels from the end, so my first attempt, wrapping the tag in `>> tool <<`\n\n, showed up as `>> tool…`\n\nwith the closing marker eaten. The kind of bug you only catch by testing it on a real phone.\n\n## Step 6: two steps, one stashed queue\n\nA simple bot is stateless: message in, reply out. This one is not. Step one (the `/digest`\n\ncommand) fetches and shows the first story; step two fires later, when I tap a button, and needs the queue from step one. `context.user_data`\n\nis a per-user dict the library keeps between handler calls, so I park the queue there:\n\n``` php\nasync def start_digest(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:\n    if update.message is None or context.user_data is None:\n        return\n    topic = context.args[0].lower() if context.args else DEFAULT_TOPIC\n    await update.message.reply_text(f\"Fetching today's {topic} stories...\")\n    try:\n        context.user_data[\"queue\"] = await fetch_stories(topic)\n    except httpx.HTTPStatusError:\n        await update.message.reply_text(\n            f\"No feed for '{topic}'. Try a Lobsters tag like rust, python, or go.\"\n        )\n        return\n    except httpx.RequestError:\n        await update.message.reply_text(\n            \"Couldn't reach Lobsters right now — try again in a bit.\"\n        )\n        return\n    await show_next(update.message, context)\n\nasync def show_next(message: Message, context: ContextTypes.DEFAULT_TYPE) -> None:\n    queue: list[Story] = (context.user_data or {}).get(\"queue\", [])\n    if not queue:\n        await message.reply_text(\"Inbox zero. That's all the trends today.\")\n        return\n    story = queue[0]\n    suggested = await context.bot_data[\"classifier\"].tag(story)\n    await message.reply_text(\n        f\"{story.title}\\n{story.url}\",\n        reply_markup=triage_keyboard(suggested),\n    )\n```\n\n`context.args`\n\nis whatever followed the command: `/digest python`\n\ngives `[\"python\"]`\n\n, a bare `/digest`\n\ngives `[]`\n\nand falls back to `DEFAULT_TOPIC`\n\n.\n\nA typo'd topic is a 404 from Lobsters, so I catch `HTTPStatusError`\n\nand reply with a helpful message, otherwise the user would just stare at a digest that never arrives. Validate at the boundary where untrusted input enters.\n\nThe second `except`\n\ncovers the other failure mode: the request never gets an HTTP response at all. `HTTPStatusError`\n\nonly fires once Lobsters answers with a 4xx/5xx — a connect timeout, read timeout, or DNS failure is an `httpx.RequestError`\n\n, which is a *sibling* of `HTTPStatusError`\n\n, not a subclass. Miss it and a flaky network crashes the handler with a traceback instead of a friendly reply. Catching `RequestError`\n\ncovers every transport-level failure (`ConnectTimeout`\n\n, `ReadTimeout`\n\n, `ConnectError`\n\n) in one branch.\n\nRead `user_data`\n\nwith `.get(...)`\n\n, never `[...]`\n\n. It lives in memory, so if the bot restarts mid-flow the dict is empty and you want a graceful reply, not a `KeyError`\n\n.\n\n`context.bot_data`\n\nis its per-bot sibling: one dict shared across all users. That makes it the right home for the classifier, which holds no per-user state. We build it once in step 7 and read it back here, so every story reuses the same OpenAI client instead of constructing a fresh one each time.\n\n## Step 7: the callback, then wire it up\n\nWhen I tap a button Telegram sends a callback query, not a message. Three rules keep it sane, numbered in the code:\n\n``` php\nasync def on_tag(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:\n    query = update.callback_query\n    if query is None or query.data is None:\n        return\n    await query.answer()  # 1. stop the spinner, first thing\n    _, tag = query.data.split(\":\", 1)  # 2. \"tag:read\" -> \"read\"\n    queue = (context.user_data or {}).get(\"queue\", [])\n    if not queue:  # stale button after a restart\n        await query.edit_message_text(\"Session expired, send /digest again.\")\n        return\n    story = queue.pop(0)\n    if tag != \"skip\":\n        save_to_reading_list(story, tag)  # the human's final say\n    await query.edit_message_text(  # 3. edit, don't reply\n        f\"Filed under {tag}: {story.title}\"\n        if tag != \"skip\"\n        else f\"Skipped: {story.title}\"\n    )\n    if isinstance(query.message, Message):\n        await show_next(query.message, context)\n```\n\nCall `await query.answer()`\n\nfirst or the loading spinner on the button never stops, even when everything else works. Edit the original message instead of replying, or the dead keyboard sits there inviting a second tap on a story you already filed. The same `.get(...)`\n\n-not-`[...]`\n\nrule applies here: an old keyboard from before a restart can still send a tap, and you want a \"send /digest again\" nudge, not a `KeyError`\n\n.\n\nRouting is by prefix. The `pattern=\"^tag:\"`\n\nis why a future second keyboard (say `setcurrency:EUR`\n\n) would not trip this handler:\n\n``` php\nasync def on_error(update: object, context: ContextTypes.DEFAULT_TYPE) -> None:\n    logger.exception(\"Handler failed\", exc_info=context.error)\n\ndef main() -> None:\n    logging.basicConfig(\n        format=\"%(asctime)s %(name)s %(levelname)s %(message)s\",\n        level=logging.INFO,\n    )\n    logger.info(\"Starting trend triage bot, polling for updates\")\n    app = Application.builder().token(config(\"TELEGRAM_BOT_TOKEN\")).build()\n    app.bot_data[\"classifier\"] = _build_classifier()\n    app.add_handler(CommandHandler(\"digest\", start_digest))\n    app.add_handler(CallbackQueryHandler(on_tag, pattern=\"^tag:\"))\n    app.add_error_handler(on_error)\n    app.run_polling()\n\nif __name__ == \"__main__\":\n    main()\n```\n\nThree things to notice here in `main()`\n\n:\n\n-\n`logging.basicConfig(...)`\n\nturns on output:`run_polling()`\n\nblocks silently otherwise, so without it a freshly started bot looks dead in the terminal even though it's happily polling. -\n`app.bot_data[\"classifier\"]`\n\nbuilds the OpenAI client once instead of per story. -\n`add_error_handler`\n\nmeans a network blip or a rate limit gets logged through`on_error`\n\nrather than vanishing into the framework.\n\n## Run it\n\n``` bash\n$ export OPENAI_API_KEY=sk-proj-...\n$ export TELEGRAM_BOT_TOKEN=...\n$ uv run trend_triage_bot.py\n2026-06-01 13:19:42,866 __main__ INFO Starting trend triage bot, polling for updates\n2026-06-01 13:19:43,190 httpx INFO HTTP Request: POST https://api.telegram.org/bot.../getMe \"HTTP/1.1 200 OK\"\n2026-06-01 13:19:43,242 httpx INFO HTTP Request: POST https://api.telegram.org/bot.../deleteWebhook \"HTTP/1.1 200 OK\"\n2026-06-01 13:19:43,244 telegram.ext.Application INFO Application started\n```\n\nOpen your bot in Telegram and send `/digest`\n\nfor the default topic, or use a tag like `/digest python`\n\n, `/digest rust`\n\nor any [Lobsters tag](https://lobste.rs/tags).\n\nThe bot walks you through today's stories one at a time. Tap the highlighted tag to accept the model's guess, or any other tag to overrule it, until you hit inbox zero:\n\nSame bot, any topic: finish the Rust queue, then `/digest python`\n\nand triage that, no code change:\n\nFiled stories land in `reading_list.jsonl`\n\n:\n\n```\n{\"tag\": \"read\", \"title\": \"One year of Roto, the compiled scripting language for Rust\", \"url\": \"https://blog.nlnetlabs.nl/one-year-of-roto-the-compiled-scripting-language-for-rust/\"}\n{\"tag\": \"lib\", \"title\": \"Announcing Rust 1.96.0\", \"url\": \"https://blog.rust-lang.org/2026/05/28/Rust-1.96.0/\"}\n{\"tag\": \"read\", \"title\": \"What kache actually caches\", \"url\": \"https://kunobi.ninja/blog/what-kache-actually-caches\"}\n{\"tag\": \"read\", \"title\": \"Creusot helps you prove your Rust code is correct\", \"url\": \"https://github.com/creusot-rs/creusot/tree/master\"}\n{\"tag\": \"tool\", \"title\": \"uv must be installed to build a standalone Python distribution\", \"url\": \"https://github.com/astral-sh/python-build-standalone/commit/c9c40c56eb53136587f0a32382cad9e5cd8d184a\"}\n{\"tag\": \"tool\", \"title\": \"SPy: an interpreter and a compiler for a statically typed variant of Python\", \"url\": \"https://github.com/spylang/spy\"}\n{\"tag\": \"read\", \"title\": \"Opaque Types in Python\", \"url\": \"https://blog.glyph.im/2026/05/opaque-types-in-python.html\"}\n{\"tag\": \"read\", \"title\": \"uv is fantastic, but its package management UX is a mess\", \"url\": \"https://www.loopwerk.io/articles/2026/uv-ux-mess/\"}\n```\n\nThat file is the actual output; one JSON object per line, ready to feed into whatever reads it next. You can find the repo [here](https://github.com/bbelderbos/telegram-hitl).\n\nThe interesting question is not whether the model can tag a headline. It's pretty accurate, but it can get it wrong, and that's where you want to have a human in the loop. This has been a simple example to show the flow, but real workflows might involve more interesting things like approving trades, triaging support tickets, or moderating content. The model can do the heavy lifting of making a guess, but the human gets the final say, and that's where the value is.\n\n## Keep reading\n\nMost AI tutorials end at \"call the API.\" This cohort ends with a deployed agent: function calling, structured outputs, three interfaces, Docker, 95%+ test coverage. Six weeks of real engineering, not notebooks. [Join the next Agentic AI cohort →](https://pythonagenticai.com)", "url": "https://wpnews.pro/news/ai-human-in-the-loop-news-digest-triage-telegram-bot", "canonical_source": "https://belderbos.dev/blog/human-in-the-loop-telegram-bot-python/", "published_at": "2026-06-01 00:00:00+00:00", "updated_at": "2026-06-17 10:01:42.344004+00:00", "lang": "en", "topics": ["ai-agents", "natural-language-processing", "developer-tools"], "entities": ["Telegram", "OpenAI", "Lobste.rs", "BotFather", "Python", "Pydantic"], "alternates": {"html": "https://wpnews.pro/news/ai-human-in-the-loop-news-digest-triage-telegram-bot", "markdown": "https://wpnews.pro/news/ai-human-in-the-loop-news-digest-triage-telegram-bot.md", "text": "https://wpnews.pro/news/ai-human-in-the-loop-news-digest-triage-telegram-bot.txt", "jsonld": "https://wpnews.pro/news/ai-human-in-the-loop-news-digest-triage-telegram-bot.jsonld"}}