Inside an AI Software Factory Dashboard for Autonomous Coding Agents

A fully autonomous "dark factory" that auto-merged coding-agent output straight to production proved reliable only for narrow cases such as proof-of-concepts, according to a goremy.ai account of building an AI software factory dashboard, and adding targeted human checkpoints made the pipeline trustworthy enough for real use. Every coding-agent run in the factory executes through Archon, an open-source workflow and harness builder that manages parallel execution using git worktrees, and all agent activity — tool calls, turns and decisions — is logged to a Postgres database via Neon, whose branching lets the entire factory configuration be rolled back to an earlier version. A scorecard view tracks per-version metrics such as lap counts, merge counts and runtime to decide whether and how to improve the pipeline.

Inside an AI Software Factory Dashboard for Autonomous Coding Agents How builders track autonomous coding agents with a mission-control dashboard covering workflows, decisions, failures, and version rollbacks. What is an AI software factory dashboard? An AI software factory dashboard is a control panel for monitoring and steering a pipeline of autonomous coding agents that plan, build, test, and ship software https://goremy.ai/blog/articles/ai-software-factory-vs-saas-why-build-beats-buy-in-2026 with minimal human intervention. Instead of tracking agent activity through scattered GitHub issues and pull requests, the dashboard gives a single view of every workflow step, every decision point that needs human approval, and every version of the factory itself, so the person running it can see the system’s shape and evolve it over time. TL;DR - A software factory takes a high-level input like a PRD or issue ticket and autonomously splits it into tasks, implements each one, verifies the code, and routes only select decisions back to a human. - The earliest version of this approach was a fully autonomous “dark factory” with zero human involvement, including auto-merging to production, which proved reliable only for narrow cases like proof-of-concepts. - Dialing back full autonomy and adding targeted human checkpoints turned out to be what actually made the pipeline trustworthy enough for real use. - The dashboard’s core value is observability : a visual map of the entire workflow, every skill the agents run, and a log of which design decisions were tried and why. - Every coding agent run in the factory executes through Archon , an open-source workflow/harness builder that manages parallel execution using git worktrees. - All agent activity tool calls, turns, decisions gets logged to a Postgres database via Neon , which also enables branching so the entire factory configuration can be rolled back to an earlier version. - A scorecard view tracks metrics per factory version, like lap counts, merge counts, and runtime, to decide whether and how to improve the pipeline. 1,284 37 6 | Vendor | Tier | Risk | Approval <sup NEW</sup | |---|---|---|---| | Northwind Supply | Tier 1 | Low | Approved | | Acme Logistics | Tier 2 | Medium | Approve | | Helios Data | Tier 1 | High | Pending | | Granite Freight | Tier 3 | Low | Approved | | Pinecrest Labs | Tier 2 | Low | Approve | How does an autonomous coding pipeline actually work? The pipeline starts with an input: a single issue ticket or an entire release described in a PRD. The factory breaks that input into individual issues, then runs each one through a sequence of steps, often referred to as “laps.” Each lap can include planning, implementation, verification, and code review, with the depth of the process adjusted dynamically based on how complex the task is. Simple tickets move through a lighter set of steps; complex ones get a fuller build-and-validate cycle so the system doesn’t waste effort on trivial changes or under-test risky ones. Multiple issues can run in parallel, which is where infrastructure like git worktrees becomes necessary, letting several agent sessions work on isolated copies of the codebase at once without stepping on each other. The factory also handles things like dependency mapping between tasks, staging environment management, and deciding where in the sequence a human needs to weigh in before work continues. Why did a fully autonomous “dark factory” fail? The first version of this system had no human checkpoints at all. Given a PRD, it would split the work into issues, implement and verify each one, and merge straight to production, no review, no approval gate, nothing. That is close to the idealized version of generative AI coding: idea in, shipped and validated code out, with zero manual steps anywhere, not even the merge. In practice, that level of autonomy only held up for narrow cases, like quick proof-of-concepts. For anything meant to run in production, full autonomy introduced too much risk. The fix wasn’t scrapping the automation, it was reintroducing human oversight at specific points, renaming the system from “dark factory” to “software factory” to reflect that shift. The guiding principle became finding the minimal set of decisions that still need a human, while keeping everything else running on its own. What goes into the dashboard? The dashboard is organized around a few recurring views: A high-level visualization of the entire factory: every workflow, every decision point, every skill the agents can call. Before this existed, it was difficult to mentally map the shape of the system, which made it hard to know where to intervene or improve things. A key decisions log that records every meaningful change made to the factory over time, what was tried, what worked, what’s currently being tested. This prevents re-running failed experiments and gives a history to compare against. Each decision ties to a specific commit, so two versions of the factory can be diffed directly, showing exactly what changed in workflows, prompts, and skills between them. Health and failure tracking , logging what broke and why. Causes mentioned include hitting rate limits on the coding model and flaws in the workflow logic itself that needed adjustment after running across hundreds of laps. Live run tracking , showing ongoing laps in progress for example, a pull request mid-review, or two tickets currently in implementation , plus a view of which issues need human attention versus which are proceeding autonomously. regulatory compliance and security. A scorecard per factory version , covering how many laps have run, how many merges have landed in that codebase, and how long that version has been active. These numbers are what determine whether a version needs further tuning. Regression and feature test tracking , since the factory generates its own test scenarios as it builds, and not every scenario runs for every PR. The dashboard shows which tests were selected and their pass/fail outcomes. Version rollback “rewind” , built using database branching, which allows reverting the entire factory setup to a prior version or testing against an older configuration without disrupting the live branch. How does Archon fit into the architecture? Archon is described as an open-source harness builder: a layer that turns any AI coding process into a reusable, observable workflow. Every coding agent invocation in the factory runs through an Archon workflow, not as a standalone script. Archon handles the harder infrastructure problems underneath, like managing git worktrees so parallel work streams don’t collide. The workflows themselves are organized into something referred to as an SDLC pack, a collection of pre-built workflows covering planning, implementation, and validation steps across the software development lifecycle. Because everything routes through Archon, there’s a consistent, inspectable record: every run, down to individual logs, tool calls, and agent decisions, is visible either in the Archon UI directly or linked to from the main dashboard. Why store everything in Postgres via Neon? Every agent turn, tool call, and decision gets written to a Postgres database, with Neon as the hosting layer. This isn’t just for audit purposes, though full audit trails matter when something goes wrong and needs a retroactive explanation. The database also becomes the foundation for improvement: without a durable record of what each agent did and why, there’s no reliable way to diagnose failures or decide what to change in the next version of the factory. Neon’s branching feature extends this further, letting the whole factory state be branched the same way code is branched, which is what enables the dashboard’s rollback feature. A past version of the factory isn’t just a log entry, it’s a reconstructable state that can be returned to and tested against. Is building a dashboard like this worth it? For a team running a handful of coding agents on small tasks, a dashboard layer is probably overkill, GitHub’s native tools are enough. The case for building one shows up at scale: once a factory is running hundreds of pull requests a week across multiple codebases, tracking everything through issue comments and raw merge history stops being workable. At that point, a dashboard is less a convenience and more a requirement for maintaining control over a system that’s otherwise too complex to hold in your head. The pattern described here a visual factory map, a decision log, failure tracking, and version diffing reflects a broader shift among teams building autonomous coding pipelines at scale: build the automation first, then build the instrumentation to manage it once it gets too complex to track manually. Frequently Asked Questions What does “human in the loop” mean in an AI software factory? Enterprises have Remy. It means specific decision points are deliberately routed to a person instead of being fully automated, such as approving a pull request merge or resolving an ambiguous requirement, while everything else planning, coding, testing runs autonomously. What is Archon in this context? Archon is an open-source workflow and harness builder that turns AI coding processes into reusable, trackable workflows. It manages execution details like parallel git worktrees and logs every agent action, and it serves as the execution layer underneath the software factory. Why use Neon and Postgres for an autonomous coding system? Postgres stores a full record of every agent decision, tool call, and turn, which supports debugging and auditing. Neon’s database branching additionally allows the whole factory configuration to be versioned and rolled back, similar to how code branches work in git. Why did the fully autonomous “dark factory” get scaled back? A system with zero human checkpoints, including automatic merges to production, proved reliable only for narrow use cases like proof-of-concepts. Reintroducing human approval at select points made the pipeline trustworthy enough for broader production use. How does the factory decide how much process to apply to a task? It evaluates the complexity of each issue and adjusts the depth of planning, implementation, and validation steps accordingly, applying lighter processes to simple tickets and more thorough build-and-validate cycles to complex ones.