# I Fed My Messiest Bug Reports to a Free AI Triage Bot. Here's the Decision Table. > Source: > Published: 2026-08-29 10:45:11+00:00 Every maintainer knows the ritual. You open the queue at 8 AM, and fifteen tabs are the same bug. One has no logs. Another has a title that reads like a ransom note. The third is a feature request wearing a bug costume. I was losing my mornings to that queue. So I ran a small experiment. Could a free AI sort the mess before I looked at it? No paid APIs. No GPU budget. Just free model access and a free server option. This article is the runbook. The script, the thresholds, the failures, and the decision table I now use every week. If you maintain anything with an issue tracker, steal all of it. Triage is pattern matching. Duplicate titles. Missing stack traces. Feature requests disguised as bugs. Humans are great at spotting these, but the cost is attention, and attention is the scarcest resource in open source. A triage bot does not need to be perfect. It needs to be cheap, predictable, and loud when it is wrong. Free AI is a good fit, as long as you label your confidence. The experiment had one rule: the bot routes, humans decide. Nothing gets closed automatically. I limited the classifier to four outputs. Fewer buckets means fewer excuses for a wrong guess. | Bucket | What it means | Example signal | |---|---|---| `needs-info` | Missing logs, steps, or version | "it crashes lol" | `likely-duplicate` | Same symptoms as a known issue | "same as #412 but on Windows" | `no-action` | Question, feature request, or praise | "can you add dark mode?" | `escalate` | Crash, data loss, security, PII | "deleted my data" | The prompt forces the model to pick one bucket and justify it in one line. Here is the entire loop. It reads a JSON export of issues, classifies each one, and writes a report. No framework, no dependencies beyond `requests` . ``` python import json import sys from pathlib import Path import requests BUCKETS = ["needs-info", "likely-duplicate", "no-action", "escalate"] def build_prompt(issue: dict, known_issues: list[dict]) -> str: known = "\n".join( f"#{i['number']}: {i['title']}" for i in known_issues[:5] ) return f""" You are a bug triage assistant. Classify the issue into exactly one bucket. Buckets: - needs-info: no logs, no reproduction steps, no version - likely-duplicate: same symptoms as a known issue - no-action: question, feature request, or praise - escalate: crash, data loss, security issue, or PII Known issues: {known} Issue title: {issue['title']} Issue body: {issue['body'][:1500]} Reply with exactly one JSON line: {{"bucket": "one of {BUCKETS}", "confidence": 0.0, "one_line_why": "..."}} """ def classify(issue: dict, endpoint: str, known_issues: list[dict]) -> dict: payload = { "model": "free", "messages": [{"role": "user", "content": build_prompt(issue, known_issues)}], "temperature": 0, } response = requests.post(endpoint, json=payload, timeout=120) response.raise_for_status() text = response.json()["choices"][0]["message"]["content"] return json.loads(text) def main() -> None: endpoint = sys.argv[1] export = json.loads(Path("issues.json").read_text()) known = [i for i in export if i["state"] == "open"] report = [] for issue in export: if issue["state"] != "open": continue try: result = classify(issue, endpoint, known) report.append({**result, "number": issue["number"]}) except Exception as error: report.append({"number": issue["number"], "bucket": "human", "confidence": 0.0, "one_line_why": str(error)}) Path("triage-report.json").write_text(json.dumps(report, indent=2)) if __name__ == "__main__": main() ``` That `"model": "free"` line is deliberate. MonkeyCode's free tier exposes free model access, a reported 10M token allowance, and a free server option. **Disclosure: This article was prepared as part of MonkeyCode's product outreach.** I am not going to tell you the quota will last forever, because quotas change. Check the current number before you schedule anything. The free server option is where the cron job lives. One batch per night, at 3 AM, when cold starts do not matter. The queue is sorted by the time I open the laptop. Confidence changes the action. This is the part people skip, and it is the part that keeps the bot honest. | Bucket | Confidence | Action | |---|---|---| `escalate` | any | Human now. Do not auto-reply. | `needs-info` | >= 0.8 | Auto-comment with a template asking for logs | `needs-info` | < 0.8 | Human queue | `likely-duplicate` | >= 0.9 | Link the oldest known issue, human confirms | `likely-duplicate` | < 0.9 | Human queue | `no-action` | >= 0.8 | Move to discussions, human confirms at weekly review | `no-action` | < 0.8 | Human queue | `human` | 0.0 | Human. The network failed, not the model. | No auto-close. Ever. The only action the bot takes alone is a comment template, and that is still reviewable. I ran the script against a synthetic fixture of twelve issues. Here are the first three so you can reproduce the run. ``` [ { "number": 1, "title": "App crashes on startup", "body": "it just crashes, nothing else. please fix.", "state": "open" }, { "number": 2, "title": "Add dark mode", "body": "Would be nice to have dark mode like the old app had.", "state": "open" }, { "number": 3, "title": "Same crash as #1 on Windows 11", "body": "Same as #1 but I am on Windows 11. Here is the stack trace: ...", "state": "open" } ] ``` The model called issue 1 `needs-info` with 0.93 confidence. Correct. Issue 2 landed in `no-action` with 0.97. Also correct. Issue 3 got `likely-duplicate` at 0.88, which sits under my 0.9 threshold, so it went to the human queue. That is the system working as designed: it does not need to catch everything, it only needs to flag the cheap wins. Three lessons survived the run. `needs-info` and `escalate` . Add the instruction that a missing body means `needs-info` , and the flip becomes boring. Boring is good.`human` row, and that is exactly what the `try/except` is for. A triage bot that silently eats issues is worse than no bot.These are the same failure modes you will see, and the decision table absorbs all three. This is not a silver bullet. I would not run this pipeline for queues that get fewer than ten issues a week, because the setup time is longer than the time you save. I would also not run it on security-sensitive content. You are sending issue bodies to an external API. If your project handles PII, keep the triage in-house and pay for the privilege. Teams that need a contractual SLA should not depend on any free tier, mine included. The script is the point. The hosting is replaceable. Finally, the bot cannot catch what the prompt does not mention. If you have a fifth bucket, add it. Keep the model at `temperature: 0` for repeatable output. Crank it up and your triage gets spicy. Do not do that. A free triage bot does not replace maintainers. It buys back the first hour of the morning. That hour is worth more than a perfect model, and you can get it for free. The fixture and the script above are yours to copy. Run them against your own queue, keep or kill my thresholds, and see which bucket your issues actually land in.