How to Build a Release Factory for AI Coding Agents A practical guide published on goremy.ai outlines a "release factory" workflow that batches tickets, runs AI coding agents through parallel planning-build-iterate loops, and stages the combined output for a single human review before deployment. The guide positions the release factory between a "dark factory" with no human review and a "PR factory" where every pull request is reviewed individually, and cites a meter.org study finding that roughly half of test-passing agent pull requests submitted to open-source projects were rejected by maintainers. It also points to Spotify's "Honk" system and Airbnb as examples of a related "fleet factory" pattern for narrower, outcome-based automation such as migrations and refactors. How to Build a Release Factory for AI Coding Agents A practical guide to the release-factory workflow: batching tickets, running coding agents in parallel, and reviewing in staging before deploy. What is a release factory, and why does it matter for AI coding agents? A release factory is a workflow for running AI coding agents on a batch of tickets at once, letting them plan, build, and iterate in parallel, then routing the finished work to a staging environment for one combined human review before anything ships to production. It sits in the middle of the autonomy spectrum https://goremy.ai/blog/articles/what-is-an-ai-software-factory-definition-and-saas-impact : more scalable than reviewing every pull request one by one, but safer than letting agents merge straight to production with no human in the loop. The core problem a release factory solves is where the bottleneck in software work actually sits now. Writing code isn’t the slow part anymore. Planning what to build and validating what got built are. A release factory batches the middle part the actual building so a person’s attention goes toward the two things that still need human judgment: scoping the work up front and reviewing it before it goes live. TL;DR - A release factory batches a group of tickets, runs coding agents through build loops in parallel, and stages everything for one combined review before deployment. - It occupies the middle ground between a fully autonomous “dark factory” no human review, high risk and a “PR factory” reviewing every single pull request, which keeps a human as the bottleneck . - The workflow still requires heavy human involvement at the edges : upfront planning and dependency mapping, then final review and sign-off before production. - Releases in this model are small and scoped , not a return to waterfall-style big-batch releases; the goal is batching enough work to be efficient without losing reviewability. - Companies like Spotify with its “Honk” system and Airbnb have shown that narrower, outcome-based automation migrations, maintenance, refactors can run with far less human review, which is a related but distinct pattern called a fleet factory . - A meter.org study found that roughly half of test-passing agent pull requests submitted to open-source projects got rejected by maintainers, a reminder that passing tests alone isn’t proof of correctness or fit. - Starting with a PR factory review every PR individually before scaling up to a release factory is a reasonable on-ramp if you’re new to running coding agents with more autonomy. › Build a loan-approval app for 2,000 bankers. already included: ✓ SOC 2✓ role-based access✓ audit log✓ self-hosted How does a release factory actually work? The workflow has three main phases, and the human stays actively involved in the first and last one. 1. Planning and dependency mapping. Before any code gets written, a person works with the agent or a planning-focused agent session to scope out a batch of tickets. This isn’t just dumping a backlog into a queue. It includes mapping dependencies between tickets so the system knows which tasks can run in parallel and which need to happen in sequence. This is also where the release gets scoped down to something reviewable. A release factory isn’t about batching an entire quarter’s roadmap; it’s about batching enough related, well-defined work that a human can still meaningfully review the output afterward. 2. Parallel build loops. Once the batch is scoped, coding agents work through the tickets in parallel, each running its own planning-build-iterate loop on its assigned piece of work. This is the part that scales: instead of one agent session per ticket with a person watching each step, multiple agents work simultaneously, and the system not the human manages that concurrency. 3. Staged review. Instead of merging each PR individually, the output of the batch lands in a staging environment. This lets a person review the combined result of multiple tickets at once, rather than context-switching between individual pull requests. Review happens here before anything goes to production, and in most setups a human still signs off before deployment. This structure explains why a release factory sits where it does on the autonomy spectrum: there’s still a human reviewing before production, but that review happens in batch rather than ticket by ticket, which is what lets the workflow scale. How is a release factory different from a PR factory? The distinction comes down to what gets batched and where the review happens. A PR factory automates the planning, building, and iterating for one ticket at a time, and a person reviews every single pull request before merge. You can still run multiple PR factory sessions in parallel on different tickets, but review stays granular: one ticket, one review, one decision. This is the most reliable setup because nothing reaches staging or production without individual scrutiny, but it also means a person is still the bottleneck in roughly the same place they’d be without a factory at all. The automation mainly removes manual orchestration between planning, building, and iterating, not the reviewer’s workload. A release factory batches multiple tickets together and defers review to a single pass in staging. This trades some of the PR factory’s granularity for throughput: a person reviews a batch of related changes at once instead of context-switching through individual PRs. The risk is marginally higher because review happens after more work has already been assembled, but the scoping and dependency-mapping step up front is meant to keep each release small and coherent enough that batch review is still manageable. Neither replaces human judgment. The difference is whether that judgment gets applied ticket by ticket or batch by batch. Enterprises build with AI on Remy. - Self-hosted - SOC 2 - HIPAA - Audit log - Role-based access How does a release factory compare to fleet factories and dark factories? It helps to think of these as points on two axes: how much a human reviews, and how confidently a passing test can prove the work is actually correct. A dark factory sits at the extreme end of autonomy: no human review, agents plan, build, and merge straight to production based on a loosely structured request. The problem isn’t the capability of the underlying model, it’s that communicating intent perfectly, every time, with no human checkpoint, is unrealistic. Mistakes in communicating intent turn into bad implementations shipped straight to production. Dark factories can work well for prototypes or proof-of-concept work where the stakes of a wrong implementation are low. A fleet factory runs the same low-review, high-autonomy workflow as a dark factory, but narrows the scope to tasks where success is unambiguous and testable, things like migrations, refactors, and routine maintenance. Spotify’s internal system, called Honk, is one documented example of this pattern, built specifically around automated maintenance pull requests. Airbnb has also published work describing a large-scale automated test-file migration. These cases work because the task has a clear, testable outcome, so skipping heavy human review doesn’t carry the same risk it would for feature work involving judgment calls. A release factory doesn’t require that same testability guarantee. It’s built for work that still needs human judgment to validate, which is most everyday feature and bug-fix work, so it keeps a review step in the loop. That’s the tradeoff: fleet factories get more scale because they stick to tasks tests can fully validate; release factories get more flexibility by keeping a human in the review loop, at the cost of needing that person’s time at each batch. Is a release factory worth setting up? For most teams running coding agents on everyday feature work and bug fixes, rather than pure migrations or maintenance, a release factory is a reasonable target. It avoids the two failure modes on either side: the unreliable, unreviewed output of a dark factory, and the bottleneck of reviewing every single pull request in a PR factory. The caveat is that it requires real investment in the planning and dependency-mapping step. A release factory isn’t just “queue up a bunch of tickets and let agents go.” The batch has to be scoped carefully enough that a staged review is still a manageable task, not an overwhelming pile of changes nobody can properly evaluate. Teams that skip the planning step and just batch tickets without dependency mapping are more likely to end up with a staging environment full of changes that don’t cohere, which defeats the purpose. It’s also worth treating a PR factory as a reasonable starting point if a team or individual developer hasn’t run coding agents with this level of autonomy before. Testing the core build loop one ticket at a time first, before expanding to batched releases, is a lower-risk way to validate that the agent harness and review process actually work before scaling up. Frequently Asked Questions › Build a safety-inspection app for 60 plants. already included: ✓ SOC 2✓ role-based access✓ audit log✓ self-hosted What’s the main risk of a release factory compared to reviewing every PR individually? Review happens later and in batch, so there’s more assembled work on the table by the time a human looks at it. The mitigation is scoping releases small and mapping dependencies carefully up front, so the batch stays coherent and reviewable rather than becoming an unmanageable pile of changes. Does a release factory mean going back to big, infrequent releases like waterfall development? No. The releases in this model are meant to stay small and tightly scoped. The batching is about grouping a handful of related tickets to review together, not accumulating a large backlog of work into one giant, infrequent release. Can a release factory use the same agents as a fleet factory? The underlying coding agents and build loops can be similar, but the workflows differ in scope and review intensity. Fleet factories work best on narrowly defined, testable tasks like migrations or maintenance where a passing test is enough proof. Release factories are built for broader feature and bug-fix work that still needs human judgment, so they keep a staged review step that fleet factories skip. How do I know if my tickets are better suited to a fleet factory or a release factory? Ask whether a passing test can confidently prove the work is correct and done. If yes, that’s a candidate for fleet-factory-style automation with minimal review. If the task involves judgment calls about product direction, design, or intent that a test can’t fully capture, it belongs in a release factory with human review before production. Is it true that AI coding agents can’t be trusted with autonomy in open-source or production codebases? Not universally. A widely cited meter.org study found about half of test-passing agent pull requests to open-source projects were rejected by maintainers, usually because the agent misunderstood project direction or intent, not because the code failed. That’s consistent with keeping judgment-heavy work inside a reviewed workflow like a release or PR factory, while reserving fully autonomous, low-review setups for tasks with clear, testable outcomes.