{"slug": "i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-here-s-the-decision-table", "title": "I Fed My Messiest Bug Reports to a Free AI Triage Bot. Here's the Decision Table.", "summary": "A developer built a free AI-powered bug triage bot that classifies GitHub issues into four buckets—needs-info, likely-duplicate, no-action, and escalate—using MonkeyCode's free tier. The bot routes issues but leaves final decisions to humans, and the developer shared the full script and decision table for maintainers to adopt.", "body_md": "Every maintainer knows the ritual. You open the queue at 8 AM, and fifteen tabs are the same bug. One has no logs. Another has a title that reads like a ransom note. The third is a feature request wearing a bug costume.\n\nI was losing my mornings to that queue. So I ran a small experiment. Could a free AI sort the mess before I looked at it? No paid APIs. No GPU budget. Just free model access and a free server option.\n\nThis article is the runbook. The script, the thresholds, the failures, and the decision table I now use every week. If you maintain anything with an issue tracker, steal all of it.\n\nTriage is pattern matching. Duplicate titles. Missing stack traces. Feature requests disguised as bugs. Humans are great at spotting these, but the cost is attention, and attention is the scarcest resource in open source.\n\nA triage bot does not need to be perfect. It needs to be cheap, predictable, and loud when it is wrong. Free AI is a good fit, as long as you label your confidence.\n\nThe experiment had one rule: the bot routes, humans decide. Nothing gets closed automatically.\n\nI limited the classifier to four outputs. Fewer buckets means fewer excuses for a wrong guess.\n\n| Bucket | What it means | Example signal |\n|---|---|---|\n`needs-info` |\nMissing logs, steps, or version | \"it crashes lol\" |\n`likely-duplicate` |\nSame symptoms as a known issue | \"same as #412 but on Windows\" |\n`no-action` |\nQuestion, feature request, or praise | \"can you add dark mode?\" |\n`escalate` |\nCrash, data loss, security, PII | \"deleted my data\" |\n\nThe prompt forces the model to pick one bucket and justify it in one line.\n\nHere is the entire loop. It reads a JSON export of issues, classifies each one, and writes a report. No framework, no dependencies beyond `requests`\n\n.\n\n``` python\nimport json\nimport sys\nfrom pathlib import Path\n\nimport requests\n\nBUCKETS = [\"needs-info\", \"likely-duplicate\", \"no-action\", \"escalate\"]\n\ndef build_prompt(issue: dict, known_issues: list[dict]) -> str:\n    known = \"\\n\".join(\n        f\"#{i['number']}: {i['title']}\" for i in known_issues[:5]\n    )\n    return f\"\"\"\nYou are a bug triage assistant. Classify the issue into exactly one bucket.\n\nBuckets:\n- needs-info: no logs, no reproduction steps, no version\n- likely-duplicate: same symptoms as a known issue\n- no-action: question, feature request, or praise\n- escalate: crash, data loss, security issue, or PII\n\nKnown issues:\n{known}\n\nIssue title: {issue['title']}\nIssue body: {issue['body'][:1500]}\n\nReply with exactly one JSON line:\n{{\"bucket\": \"one of {BUCKETS}\", \"confidence\": 0.0, \"one_line_why\": \"...\"}}\n\"\"\"\n\ndef classify(issue: dict, endpoint: str, known_issues: list[dict]) -> dict:\n    payload = {\n        \"model\": \"free\",\n        \"messages\": [{\"role\": \"user\", \"content\": build_prompt(issue, known_issues)}],\n        \"temperature\": 0,\n    }\n    response = requests.post(endpoint, json=payload, timeout=120)\n    response.raise_for_status()\n    text = response.json()[\"choices\"][0][\"message\"][\"content\"]\n    return json.loads(text)\n\ndef main() -> None:\n    endpoint = sys.argv[1]\n    export = json.loads(Path(\"issues.json\").read_text())\n    known = [i for i in export if i[\"state\"] == \"open\"]\n    report = []\n\n    for issue in export:\n        if issue[\"state\"] != \"open\":\n            continue\n        try:\n            result = classify(issue, endpoint, known)\n            report.append({**result, \"number\": issue[\"number\"]})\n        except Exception as error:\n            report.append({\"number\": issue[\"number\"], \"bucket\": \"human\", \"confidence\": 0.0, \"one_line_why\": str(error)})\n\n    Path(\"triage-report.json\").write_text(json.dumps(report, indent=2))\n\nif __name__ == \"__main__\":\n    main()\n```\n\nThat `\"model\": \"free\"`\n\nline is deliberate. MonkeyCode's free tier exposes free model access, a reported 10M token allowance, and a free server option. **Disclosure: This article was prepared as part of MonkeyCode's product outreach.** I am not going to tell you the quota will last forever, because quotas change. Check the current number before you schedule anything.\n\nThe free server option is where the cron job lives. One batch per night, at 3 AM, when cold starts do not matter. The queue is sorted by the time I open the laptop.\n\nConfidence changes the action. This is the part people skip, and it is the part that keeps the bot honest.\n\n| Bucket | Confidence | Action |\n|---|---|---|\n`escalate` |\nany | Human now. Do not auto-reply. |\n`needs-info` |\n>= 0.8 | Auto-comment with a template asking for logs |\n`needs-info` |\n< 0.8 | Human queue |\n`likely-duplicate` |\n>= 0.9 | Link the oldest known issue, human confirms |\n`likely-duplicate` |\n< 0.9 | Human queue |\n`no-action` |\n>= 0.8 | Move to discussions, human confirms at weekly review |\n`no-action` |\n< 0.8 | Human queue |\n`human` |\n0.0 | Human. The network failed, not the model. |\n\nNo auto-close. Ever. The only action the bot takes alone is a comment template, and that is still reviewable.\n\nI ran the script against a synthetic fixture of twelve issues. Here are the first three so you can reproduce the run.\n\n```\n[\n  {\n    \"number\": 1,\n    \"title\": \"App crashes on startup\",\n    \"body\": \"it just crashes, nothing else. please fix.\",\n    \"state\": \"open\"\n  },\n  {\n    \"number\": 2,\n    \"title\": \"Add dark mode\",\n    \"body\": \"Would be nice to have dark mode like the old app had.\",\n    \"state\": \"open\"\n  },\n  {\n    \"number\": 3,\n    \"title\": \"Same crash as #1 on Windows 11\",\n    \"body\": \"Same as #1 but I am on Windows 11. Here is the stack trace: ...\",\n    \"state\": \"open\"\n  }\n]\n```\n\nThe model called issue 1 `needs-info`\n\nwith 0.93 confidence. Correct. Issue 2 landed in `no-action`\n\nwith 0.97. Also correct. Issue 3 got `likely-duplicate`\n\nat 0.88, which sits under my 0.9 threshold, so it went to the human queue. That is the system working as designed: it does not need to catch everything, it only needs to flag the cheap wins.\n\nThree lessons survived the run.\n\n`needs-info`\n\nand `escalate`\n\n. Add the instruction that a missing body means `needs-info`\n\n, and the flip becomes boring. Boring is good.`human`\n\nrow, and that is exactly what the `try/except`\n\nis for. A triage bot that silently eats issues is worse than no bot.These are the same failure modes you will see, and the decision table absorbs all three.\n\nThis is not a silver bullet. I would not run this pipeline for queues that get fewer than ten issues a week, because the setup time is longer than the time you save.\n\nI would also not run it on security-sensitive content. You are sending issue bodies to an external API. If your project handles PII, keep the triage in-house and pay for the privilege.\n\nTeams that need a contractual SLA should not depend on any free tier, mine included. The script is the point. The hosting is replaceable.\n\nFinally, the bot cannot catch what the prompt does not mention. If you have a fifth bucket, add it. Keep the model at `temperature: 0`\n\nfor repeatable output. Crank it up and your triage gets spicy. Do not do that.\n\nA free triage bot does not replace maintainers. It buys back the first hour of the morning. That hour is worth more than a perfect model, and you can get it for free.\n\nThe fixture and the script above are yours to copy. Run them against your own queue, keep or kill my thresholds, and see which bucket your issues actually land in.", "url": "https://wpnews.pro/news/i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-here-s-the-decision-table", "canonical_source": "https://dev.to/devgo_7763/i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-heres-the-decision-table-5jc", "published_at": "2026-08-29 10:45:11+00:00", "updated_at": "2026-08-29 11:19:07.566623+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "natural-language-processing"], "entities": ["MonkeyCode"], "alternates": {"html": "https://wpnews.pro/news/i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-here-s-the-decision-table", "markdown": "https://wpnews.pro/news/i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-here-s-the-decision-table.md", "text": "https://wpnews.pro/news/i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-here-s-the-decision-table.txt", "jsonld": "https://wpnews.pro/news/i-fed-my-messiest-bug-reports-to-a-free-ai-triage-bot-here-s-the-decision-table.jsonld"}}