{"slug": "ask-it-again-how-we-review-ai-code-in-ten-passes", "title": "Ask It Again: How We Review AI Code in Ten Passes", "summary": "A team building the PagibleAI CMS codebase, which spans more than 25 packages with about 2,400 PHP tests and 2,100 frontend tests, reviews AI-generated code with ten sequential narrow review prompts run repeatedly until each finds nothing more, using Claude Code with Claude Opus 5.5. The approach surfaced 15 gaps in a new Joomla importer against Joomla's documentation and ten more after those were fixed, including one that gave menu-linked articles the wrong URL and would have broken old links and backlinks. An Architecture pass on the 2,800-line Joomla import command split conversion, URL resolution and access decisions into separate classes, cutting the command to about 1,300 lines with all 140 Joomla tests passing.", "body_md": "We use AI heavily in our projects, e.g. [PagibleAI CMS](https://pagible.com), both for new features and for rewriting existing code. Usually, its first output is average at best. The code works, but it misses requirements, duplicates logic or puts code in the wrong layer. When we first compared our new Joomla importer with Joomla’s documentation, the agent found 15 gaps, and after fixing them, ten more. One of them gave menu-linked articles the wrong URL, which would have broken old links and backlinks.\n\nOver the past months, we’ve built and refined a list of review prompts that we run one by one, each until it finds nothing more, to turn such a first draft into good code that fits our architecture and guidelines.\n\nThe [codebase](https://github.com/aimeos/pagible) consists of more than 25 packages, about 2,400 PHP tests and 2,100 frontend tests. We use Claude Code with Claude Opus 5.5 every day for code, tests and reviews. The examples are in PHP, but the prompts work for any language.\n\nA narrow prompt makes the agent check one concern across all the changed code. Its findings stay on topic, even if many are weak, while broad requests produce more guesses. Because one fix can undo an earlier one, we repeat each prompt instead of running it once.\n\nMany weaknesses of a first draft come from the agent not knowing the project’s rules. A context file with those rules prevents most of them before any review. Our AGENTS.md has about 170 lines, and the agent reads it at the start of each session. The rules are short and use the names from the code:\n\n```\n**Security**: Structured input is schema-validated;HTML uses HTMLPurifier; SVG uploads are sanitized;remote fetches validate and pin resolved hosts to mitigate SSRF\n```\n\nThe rest covers tenancy, caching, coding style and testing. Each rule replaces a correction we used to repeat in every session, and it gives the reviews a reference for accepting or rejecting a finding.\n\nWe run these passes for every change, even small ones. By default, the agent compares the feature branch with main and checks the changed code and the surrounding code it touches.\n\nImage by the author\n\nArchitecture comes first because a structural change can make earlier fixes worthless. Security, the broad review and the tests come last because they need the final code, and by then the broad review has less noise to report. After the ten passes, the agent runs the full test suite and static analysis until both are clean.\n\nThe agent lists its findings and recommends which ones are worth implementing. The lists are long, so the recommendation helps us sort them. We usually follow it, but we decide. A bug or security finding counts if we can reproduce it and it contradicts a requirement, or if a test fails without the fix. A performance finding counts if it removes real work on a path that runs often, and a structural one if it fits our architecture, our project rules and the goal of the change. We ignore suggestions that only change style.\n\nThe agent applies the confirmed fixes, we read the diff, and we repeat the pass until no confirmed issue remains. For the Drupal importer, the Architecture pass took four rounds: the first two found changes worth making, the third nothing substantial and the fourth nothing at all. If a later fix changes code an earlier pass reviewed, we run that pass again.\n\nIn the Joomla importer, the Architecture pass found that the 2,800-line import command mixed converting content, resolving URLs, deciding access and saving pages. That broke our rule that commands save pages and everything else only reads or decides. The agent moved the first three jobs into their own classes, and the next round split off one more. The command went down to about 1,300 lines, and all 140 Joomla tests passed after the split. The round after that found nothing more, because further splitting would only add wiring.\n\nOnce, the security pass reported a medium-severity problem: the blog list shows articles without checking their access rules. That is intentional. Teasers of restricted articles are meant to be visible, and only the article itself is protected. We could reproduce it, but it contradicted no requirement. The agent argued its case well, but it didn’t know this rule.\n\nPaste them one at a time, in the order of the list above, into the same conversation, so the agent knows the earlier findings and our decisions and doesn’t suggest a change we already rejected. The first prompt asks for simplifications through a different architecture, which is what makes it an architecture pass.\n\nWithout the “non-low” limit in the performance and security prompts, we would reject about every second finding. For larger work, we replace “the changed code” with the area, like “the code related to the Joomla importer”, so the agent reviews the whole component.\n\nIn Claude Code, you can save each prompt as a custom command, for example .claude/commands/reduce.md, and run it with /reduce. If your agent doesn't have /code-review, use its broad review feature or define an equivalent command.\n\nA small change needs one or two rounds per pass, a large one ten or more. Each round takes one to three minutes, so a small change needs 10 to 60 minutes of agent time, not counting our review or the test matrix. A large change can take an afternoon.\n\nThe passes don’t look into the libraries the code uses. Our CMS pins remote fetches to the checked IP, so a malicious DNS server can’t redirect them to internal addresses. A security researcher found that streamed downloads used a different library handler that ignored the pin. The [fix](https://github.com/aimeos/pagible/commit/166191906debba6ff159367db3e156891a0a19be) uses a handler that applies the pin, and its test requests a host that doesn’t exist in DNS, so the response can only come from the pinned IP.\n\nThe passes check the code only against the requirements we give them, so a wrong or missing requirement goes through all ten unnoticed. That’s why we checked the Joomla importer against Joomla’s documentation separately, which found the gaps from the introduction. The passes also review one branch at a time, so they miss problems that only appear when several services work together. For those gaps, we rely on behavior tests, on reading the library code behind security-relevant calls, and on outside reports like the researcher’s.\n\nBesides the normal cases, we ask the agent to test error cases and unusual input, and we read the assertions. A good test fails with the wrong implementation, like the DNS test above.\n\nWe run the full suite on PHP 8.2 and 8.5 against SQLite, MySQL, MariaDB, PostgreSQL and SQL Server. Our page cache stored gzipped HTML, which worked on SQLite but failed on MySQL and MariaDB, where Laravel’s database cache store uses a text column that rejects binary data. The [fix](https://github.com/aimeos/pagible/commit/de7f74e31cc1f9a689537eaf2d7e21e9f6170ba7) encodes the content before storing it.\n\nThe passes turn the agent’s average first draft into code that fits our architecture and guidelines. They don’t replace tests or outside reviews. Every pass missed the DNS gap that a security researcher found.\n\nIf you run similar passes, we’d like to hear which prompts you use and which of ours you’d drop.\n\n[Ask It Again: How We Review AI Code in Ten Passes](https://pub.towardsai.net/ask-it-again-how-we-review-ai-code-in-ten-passes-26b4ebe78130) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/ask-it-again-how-we-review-ai-code-in-ten-passes", "canonical_source": "https://pub.towardsai.net/ask-it-again-how-we-review-ai-code-in-ten-passes-26b4ebe78130?source=rss----98111c9905da---4", "published_at": "2026-11-19 14:31:02+00:00", "updated_at": "2026-11-19 14:42:20.935989+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "developer-tools", "large-language-models"], "entities": ["PagibleAI CMS", "Joomla", "Claude Code", "Claude Opus 5.5", "Drupal", "Pagible"], "also_reported_by": [], "takeaway": "Run ten narrow, single-concern review prompts repeatedly until each finds nothing, and re-run any pass whose code a later fix touched — the approach cut a 2,800-line Joomla import command to about 1,300 lines with all 140 tests passing.", "recommendation": {"advice": "Keep a roughly 170-line AGENTS.md of project rules and run narrow single-concern review passes in sequence, repeating each until it finds nothing and re-running earlier passes when later fixes change their code.", "playbook": "coding-agents", "playbook_url": "https://wpnews.pro/playbooks/coding-agents"}, "alternates": {"html": "https://wpnews.pro/news/ask-it-again-how-we-review-ai-code-in-ten-passes", "markdown": "https://wpnews.pro/news/ask-it-again-how-we-review-ai-code-in-ten-passes.md", "text": "https://wpnews.pro/news/ask-it-again-how-we-review-ai-code-in-ten-passes.txt", "jsonld": "https://wpnews.pro/news/ask-it-again-how-we-review-ai-code-in-ten-passes.jsonld"}}