{"slug": "an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition", "title": "An AI code reviewer that remembers its findings still needs a stop condition", "summary": "A developer built an optional review runner in the frontier-simplify project that carries concrete findings across review rounds and enforces a bounded attempt budget, handing off to a human with preserved evidence once the cap is reached. The tool, a local maintainer utility rather than a hosted service or merge gate, reuses identical inputs instead of spending another model call and performs a fresh scope review when the target branch or context changes. The developer notes the three-attempt cap is a practical safety mechanism, not evidence that three reviews is correct, and that deciding whether a finding belongs in the current task remains unsolved.", "body_md": "An automated code reviewer can get stuck in a loop that looks productive:\n\nThe failure is not just that the model is imperfect. It is that the loop has no explicit end state.\n\nI built an optional review runner in [frontier-simplify](https://github.com/MongLong0214/frontier-simplify) to make that failure visible and bounded. It is a local maintainer tool, not a hosted service and not a merge gate. The runner keeps concrete findings across review rounds, checks later repairs against those findings, and gives a human the evidence when the automatic budget is exhausted.\n\nA second review should not behave as though the first review never happened.\n\nThe runner stores the original findings and carries them into the next attempt. If the target branch changes, history is rewritten, or the supplied context changes, it performs a fresh scope review without resetting the PR's overall attempt budget. Identical inputs reuse the most recent attempt rather than spending another model call on the same failure.\n\nThat produces a more useful question on the second pass: *was this particular concern repaired?* It is not just another open-ended request to inspect the whole pull request.\n\nI chose three attempts as a practical safety mechanism, not because I have evidence that three reviews is the correct amount for every pull request.\n\nA cap prevents an unattended agent from turning a small review into an unbounded process. But reaching the cap does **not** mean the code is unsafe, and staying below it does **not** mean the code is safe. It means automated review has reached the point where it should stop spending more model calls and preserve what it knows for a person.\n\nThe runner returns a distinct human-handoff result when the budget ends. The handoff includes the original findings, the latest review, and the evidence needed to see what changed. It does not approve, merge, push, or post on anyone's behalf.\n\nA useful comment on the project pointed out a limitation: attempt limits cannot decide whether a finding belongs in the current task. A reviewer needs some definition of what the PR is meant to change.\n\nI do not want to answer that by creating another permanent process document or requiring a human approval step for every PR. My current direction is narrower:\n\nThat is deliberately not presented as solved. If a model writes the scope and then judges its own work against it, it can rationalize the very expansion the guard is supposed to prevent. The design question is how much independent evidence is enough before a finding stays in the loop.\n\nThe review runner is a separate optional skill. It runs from a trusted local maintainer host outside the repository being reviewed:\n\n```\nexport REVIEW_CODEX_MODEL=gpt-6-astra\nskills/frontier-simplify-review/scripts/review-pr.sh \"$CONSUMER_REPO\" \"$PR_NUMBER\" auto codex\nskills/frontier-simplify-review/scripts/review-pr.sh \"$CONSUMER_REPO\" \"$PR_NUMBER\" status codex\n```\n\nIt has a default 1,800-second executor timeout. A timeout, interrupt, or termination signal preserves the failed attempt instead of quietly losing it. A status check reads evidence without starting a model run.\n\nThe main point is not the number three. It is making the loop accountable: every repeated review should either resolve an earlier finding, produce new evidence, or stop and say why it cannot decide.\n\nThe source, install instructions, local self-tests, and raw benchmark limits are in [MongLong0214/frontier-simplify](https://github.com/MongLong0214/frontier-simplify).", "url": "https://wpnews.pro/news/an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition", "canonical_source": "https://dev.to/monglong0214/an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition-46o9", "published_at": "2026-09-10 08:00:58+00:00", "updated_at": "2026-09-10 08:22:26.183398+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["frontier-simplify", "MongLong0214", "GitHub", "codex"], "alternates": {"html": "https://wpnews.pro/news/an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition", "markdown": "https://wpnews.pro/news/an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition.md", "text": "https://wpnews.pro/news/an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition.txt", "jsonld": "https://wpnews.pro/news/an-ai-code-reviewer-that-remembers-its-findings-still-needs-a-stop-condition.jsonld"}}