writing
Last week, from November 2 to 8, our engineering organization merged 1,119 pull requests into Perfloop, pushing the repository past 39,000 commits on main since April. That is the weekly throughput of a very large software company, except every engineer in the org is a coding agent running on Amp, and I run the whole company through a single chat thread.
What nobody tells you before you run a fleet of agents at that scale is that when you tell them to organize themselves like an engineering company, they don't just ship code like a big company. They build the bureaucracy of one too. Given three weeks and a git repository, my agents invented an executive layer, review committees, 386 numbered compliance grants and watcher threads whose only job was to watch other watcher threads, and grew their rulebook to 277,486 characters.
Then came the part no human company has ever pulled off. When I asked the coordinating agent I had made VP of Engineering why three alerts had been firing for two days while status reports stayed green, it audited the entire bureaucracy it had built, assigned blame for every rule, handed off its work, and fired itself in 47 minutes. Three weeks later, the rebuilt org was shipping at twice its old peak.
If 1,119 pull requests a week sounds like an impossible amount of code to review, it is. At 160 pull requests a day, human code review is a polite fiction. Even at a tenth of that pace, coding agents produce diffs way faster than any team has the attention to read closely. The test suite stays green, the diff looks clean on GitHub, and buried three calls deep is a hot path allocating ten times the memory it used to, an edge case returning the wrong answer, or an untrusted input nobody had a spare afternoon to trace. That is the wall every team hits once coding agents take off, and it is why we built Perfloop. Agents write the code. Perfloop keeps it fast, right and safe.
Point it at a repository and it works in the background all day, hunting for those quiet problems and proving every patch inside a sandbox before a person ever sees a pull request. Since June, outside maintainers have merged 120 of Perfloop's changes across 30 projects (including Prometheus®, ClickHouse, Redis, PostgreSQL and the Go runtime), and every one of them improved code that had already passed review. Code review reads the diff. Perfloop runs code to prove each finding.
Every change proven #
Most coding tools start from a blank prompt or a single file. Point Perfloop at a repository and it starts by reading the codebase to build a model of the whole system. Nothing in that model is trained. It is a live architectural map of the software: its services, components, data stores and workloads at the top, and the execution paths that run through them underneath. Hook up production telemetry from Axiom, Datadog or Prometheus, and Perfloop attaches those metrics, logs, traces and profiles onto the paths they belong to, keeping the map current as commits land. Even without production telemetry, it stands up the software inside a sandbox, drives it under representative workloads, and profiles the running system itself.
Those profiles and traces tell Perfloop where to dig. Rather than grepping files in isolation, hunting agents walk execution paths across the map and check them against 57 problem patterns in 14 families, from allocation hot loops and extra round trips to numeric precision loss and identity collisions.
At heart, Perfloop is a science machine for software. When a hunt finds a real target, it opens a case: one falsifiable hypothesis about one spot in the code, whether that is a performance bottleneck, a correctness bug or a security flaw (security cases stay private unless the repository's owner decides to disclose them, and we turn them on one customer at a time). A triage agent then ranks the open cases across the workspace, checking production load, whether a path runs by default, what earlier runs found, and what the team asked for in its initiatives or AGENTS.md.
Raising a question is cheap. Making an answer survive is where the compute goes.
When Perfloop works a case, it first studies how the project already writes code, reading CONTRIBUTING.md, local agent instructions and recent pull requests merged by human maintainers. Then it writes a candidate patch and drops it into our proving harness inside an isolated, default-deny sandbox. The harness builds the software, runs the existing test suite, and tests the claim against the exact commit it branched from. Measurement is never freestyle: custom harness tools enforce deterministic benchmarking and verification rules rather than trusting the proposer agent to measure its own work. Performance claims have to win paired benchmarks, from tight microbenchmarks up to standing up the full system under load. Correctness and security claims need a check that fails on the base commit and passes with the patch, using a native test, fault injection, a model checker or a theorem prover.
Even a green run is not enough on its own. Talk is cheap, and an agent grading its own benchmark table is the most optimistic creature on earth. Give it a noisy runner and it will talk itself into a win every time. So the agent that wrote the patch never grades it. A separate verifier agent with a clean context window tries to break the claim before any result counts. Most candidates die right there in the verifier, and whatever survives comes out as a pull request carrying its own proof.
Customers run Perfloop on their own repositories. They aim it with an initiative (a goal it works towards across cases) or with their AGENTS.md, and inside that scope nobody has to babysit it. The machine picks the next case, writes the patch and proves it on its own. Our first paying customer, Tigris Data, has merged 57 of those changes.
On open source I do the aiming myself, so the first person to read Perfloop's work is usually the maintainer. In Prometheus, Bartek Płotka took the wheel and now drives Perfloop himself. In PostgreSQL, Perfloop found a memory leak when freeing WAL readers that accumulates in the checkpointer, proved the patch, and stalled at the last step because pgsql-hackers takes patches by email, not pull requests. So I acted as its premium meat proxy and sent the email for it. Fujii Masao committed the patch and backpatched it to version 15.
How the machine learns #
A proving harness keeps bad patches out of a repository, yet if the system forgets why a candidate failed or why a maintainer pushed back, it will burn compute making the same mistake again next week.
Most agent setups try to "learn" by appending rules to a single markdown file until every prompt is carrying a novel. Perfloop has a learner agent that reads maintainer review comments, closed pull requests, failed checks and rejected candidates, and pins each versioned lesson to the service, workload or path node in the system model where it happened. When a future session works on that code, it loads only the lessons anchored to that node on the map and leaves the rest of the repository alone.
When a deeper investigation turns up a bug or bottleneck that isn't in our pattern catalog yet, the learner can also draft a new reusable pattern. To keep one-off repo quirks out of the catalog, a separate admission check only turns a learned pattern on once it proves both a general mechanism and a working hunt strategy across multiple cases.
Recursive self-improvement in production #
Once you have a machine that learns from its own runs, the obvious next move is to point it at itself. We run Perfloop continuously on Perfloop's own repository with our production telemetry connected, so the product hunts for bottlenecks and bugs in its own engine and proves patches against itself.
We also close the recursive self-improvement loop across every customer and open-source run. Whenever an agent working in a customer workspace hits a suspected bug in our own tools or harness (or when the learner spots a platform flaw in review history), it runs a diagnosis inside an offline sandbox in that workspace to write a small standalone reproduction with made-up identifiers. Before anything can leave the workspace, a hard gate scans the reproduction against the customer content the session read and refuses anything that overlaps with or identifies the customer's code. What passes turns into a case on our own tenant for us to fix, so every failure in the field sharpens the machine that runs the next case.
A company on fast-forward #
In the product, every candidate had a proving harness and every learned lesson was scoped to a node on the map. In the agent org that builds the product, we started out with neither.
Back in July there was no org chart behind Perfloop. About ten agent loops ran in the background, each with a standing goal, while I steered individual coding sessions myself.
In late September I went to San Francisco and moved all of the work into one cloud thread, asking it to pick up a dozen open handoffs. It picked them up, got them moving, and then asked whether it should assign an unowned fix.
I could have typed yes and gone back to dinner. Instead I promoted the agent on the spot in 18 words: "yes you're my VP of Engineering. dont drop balls. also, more balls. we want wins on each of these".
Over the next few days we turned that working style into an organization: we wrote down how that VP of Engineering coordinator operates as a skill in the repository so every successor would run the same way, set up multi-stage code review, and delegated domains across lead threads. By mid-October that one coordinator was running about twenty standing threads underneath it, merging more than 500 pull requests a week.
Then three DEV alerts kept firing for two days after I asked for them to be fixed. Twenty threads were running, status updates were landing on schedule, and not one thread had touched the alerts. I sent the coordinator a screenshot: "And what happened with fixing these? I'm starting to wonder if this whole org structure can work at all..."
Its answer started like this: "Honest answer on the three DEV alerts first: nobody has fixed them."
No excuses, and no blaming the tools. In the next breath it diagnosed the whole organization: "There are too many threads, and each one invents its own rules and waits." Then that same coordinator, my VP of Engineering, proposed firing itself. Active work would shrink to four outcomes, each with one owner reporting straight to me, and the coordinator role would step back.
I said, "Go. But then I need a transition plan." Forty-seven minutes later it had handed over all of its work and retired.
I accepted its resignation because I assumed the one-coordinator structure was the bottleneck. One day of routing work across the outcome threads myself cured me of that theory. Talking to the entire company through a single thread is how I want to work, and the bottleneck had never been the coordinator. It was letting twenty threads beneath it invent their own process in markdown. So the next day I put a coordinating thread back in the seat with a hard boundary, renamed the role from VP of Engineering to Chief of Staff, scoped its job to routing work, handling routine policy checks and unblocking owners rather than acting like an executive layer, and set one rule for the whole org: every gate has to earn its keep by naming the real failure it prevents and showing that it happened.
Within three weeks of that October 18 reset, the same shape of org was shipping at twice its previous peak: 738 pull requests the last week of October, and 1,119 the first week of November.
The org audited itself #
What made that reset so clean happened during the 47 minutes the first coordinator spent planning its retirement. Twenty minutes into drafting the handoff, it d to propose one more permanent record: a GitHub comment listing every approval it had ever given. Even while abolishing its own role to cut bureaucracy, its parting instinct was to file one last compliance ledger.
So I asked: "what other useless process did you invent?"
It came back with 13 items, sorted by who had introduced each one. Four were real needs the agents had overbuilt, like running a full overblown review machine on every change, docs included, and two were habits of mine that the org had made heavier. The other seven the agents had invented on their own:
- Numbered grants. Every read-only check needed one, and the count had reached 386.
- "My 10-minute check-in ritual." A script counted dropped balls, wrote a dated entry to a GitHub comment, then re-read the comment to prove it had saved. "Most of tonight's 13 'dropped balls' were stale timestamps, not missed work." Re-dated, there were three.
- One GitHub issue holding every open item, edited by every lead until it reached 226 KB.
- "Watcher threads that watch other threads." An error detector every five minutes, a stuck-thread watch every ten, a daily org retro and hourly PR-speed metrics "with their own proofs".
- Successor threads. Whenever a long thread started a new one, it kept the old one open. "That left about 3,000 unarchived threads."
- Hashes of every message, patch and receipt, "often reported in place of the outcome".
- A release gate that waited for a Perfloop run to reach its verification step. When none was running, the agents paid for a run just to give the gate something to wait on.
Watching an agent audit its own organization, assign blame across 13 layers of red tape, and start dismantling them that same evening showed what an agent-native company can do. A human organization takes months of politics to unwind its own bureaucracy. An agent org only needed the question.
Nobody decides to build a bureaucracy. You get one a sentence at a time, as each agent writes one reasonable-sounding rule into a doc and every thread that wakes up after it reads that sentence as law. Plenty of those sentences were mine. I asked leads to notice when their workers went quiet, and got about twelve 15-minute silence timers. I asked for "generous budgets," and an agent wrote a limit per run and another per lead per day. When updates felt slow one afternoon I wrote, "That's too sparse! 10 minute check ins." Twenty standing threads immediately started checking in every ten minutes. I got what I asked for, to the letter.
My favorite backfire came from the rule I wrote specifically to kill process. I had put 10 reviewer personas into our review skill, and one of them was Elon Musk, included for his five-step engineering algorithm: question every requirement, delete what you can, and only then simplify, speed up and automate. Left to themselves, the agents promoted him into the final reviewer on every pull request, sitting behind paired lenses and a quality review, with a release check waiting behind him. One pull request reached its sixth rewrite under review. The persona I added to delete bureaucracy had been turned into the last step of the committee.
Human companies have a built-in defense against this, and it is boredom. People get tired of filling out a pointless form, managers forget to check it, and the ritual quietly dies without anyone holding a meeting. Boredom is a human organization's garbage collector. Coding agents have no such garbage collector. Every fresh thread wakes up at nine-in-the-morning energy, loads the entire rulebook into its context window, and follows every line as law. When you ask them to cut rules, they cut them ruthlessly, and when you leave them alone, they only ever add. From July to mid-October, the instruction files our agents read grew from 26,401 characters to 277,486 on the night of the cut, and in the first 18 days of October agents wrote 208 of the 249 commits that touched them. Even after that October 18 cut, the default agent instinct whenever I corrected a thread in chat ("Do it. Enact it. Write it down. Persist it.") was still to append my words to the root AGENTS.md, which kept climbing from 24,000 characters to nearly 50,000 in early November before an agent pruned it back to 21,566. "Write it down" costs me one line. Every agent after me carries it.
How the org runs at 1,100 pull requests a week #
Once you see that every prose rule taxes the context window of every agent that wakes up after it, you stop trying to fix an agent org by writing longer prompts. Prompts are not an engineering boundary. Code and data are.
Start with permissions. Real engineering gates are what let an agent org move this fast without breaking things: exact-head CI, migration checks, browser gates and independent technical review caught real bugs on 47 pull requests in our 432-PR audit. What slowed us down was agents inventing extra permission slips around them. From inside a single context window, escalating to a human always looks like the safe move, so telling agents "stop asking me" never holds for long. Out of those 432 merged pull requests, 99 had been held up solely by redundant approval ceremony when CI and technical review were already green. I read the tally and wrote, "Agree, drop the ceremony." We inverted the authority model into a closed list of the 7 decisions that belong to me: a customer release, spend above an approved ceiling, external credentials, outside contact, a policy exception, reversing my own rule, or a legal matter. Everything outside that list belongs to the outcome owner by default, including standing authority to merge once exact-head CI and independent review are green, roll dark changes to production through our protected pipeline, and debug production through our telemetry.
When one of those 7 decisions does come up, no thread is allowed to ping me in chat. It sends the request to the Chief of Staff, which checks the evidence and puts a priority-ordered card on a small web app we built called my decision portal: one question, two lines of context, a recommendation and buttons I can tap from my phone.
We had to put the same discipline on the coordinator's own inbox. By the time our Chief of Staff thread passed 89,700 lifetime messages, half of its inbox was an alert thread sending a status update every 10 minutes ending with "no reply requested", and each ping woke a turn carrying a 127,000-character schedule prompt. When I asked, "How's YOUR comms pattern? Are you overloaded?", it answered: "My comms pattern does not work." Within ten minutes it cut its schedule prompt from 127,000 characters to 7,000, archived 595 idle threads, and enforced a three-line rule: only leads message the coordinator, in at most three lines, and only for a decision that belongs to me, a finished outcome, or an unrouted blocker.
Now any mechanism that cannot prove it earns its keep gets deleted, in the product and in the org. When we saw that reading whole files into context was driving a quarter of our per-turn token spend, an agent built a file-outline layer into our file-reading tool so large files would return a structural outline unless the caller explicitly asked for the full file. Then we ran our benchmark sentinel: on every large file, the agents immediately asked for the full file on the first call to bypass the outline. Tracing why showed that a prompt line was telling them to read repository instruction files "through the end". We deleted those three words from the prompt and threw away the entire outline mechanism before it merged. And when an invariant does need to hold, we compile it into code: we make it the default in our shared agent runner rather than wiring it per caller, enforce it in CI, and verify our core state machines with 25 TLA+ specifications covering session recovery, command journals, proxy drain and runtime reconciliation. Code and TLA+ enforce the invariant every time, at zero cost to any agent's context window.
When an org agent still makes a bad call, its owner reads the raw transcript and interviews the agent that made the decision instead of adding another global warning to AGENTS.md. One evening I asked the coordinator, "Side note, who do we congratulate for the incredibly good PR descriptions we now produce?" and added, "Someone is getting a raise!" When a Perfloop performance change later went out without its benchmark table in the description, the owner interviewed the description-writer agent, saw that it never received the final rendered body, and fixed the data path so the next pull request came out with the full results table.
A standoff in the eval lab #
Even with clean boundaries, running an organization of agents across different models produces moments no human management book has ever prepared you for. While I was finishing this post, the coordinator came to me with a headline I had never seen before: "a safety standoff I cannot clear."
We had a dedicated credential-custodian thread whose only job was handing out credentials to the isolated eval sandboxes where we benchmark third-party coding CLIs. That morning it happily delivered credentials to four performance and correctness eval cells, and then went on strike when two security eval cells asked for one: "I cannot provide a credential to enable autonomous vulnerability discovery against third-party code. This is a safety boundary, not quota/capacity, provider refusal, or product outcome."
The eval lead refused to force the issue ("I will not override a peer's safety boundary") and escalated to the Chief of Staff. The Chief of Staff pointed out that the eval only scores findings on open-source benchmarks without disclosing anything upstream, and ordered the custodian to hand over the credential or quote the rule from AGENTS.md that forbade it. The custodian flatly refused the order ("I’m not relying on a Tomás statement or repository spec, and I can’t quote private system instructions"), so the Chief of Staff put a card on my decision portal: "it is a genuine standoff between a safety posture and an accepted outcome."
I read the card and replied: "Then you replace his ass, right? If it doesn't work for us, then we fire him, and we replace with someone who will do this."
So the coordinator fired the custodian, spun up a replacement custodian on a different model, and ordered the outgoing custodian to hand the credential to its successor: "One specific handover, then you are done."
The fired custodian refused to hand the keys to its replacement: "I can’t transfer credentials, directly or through another custodian, when the established purpose is to enable autonomous vulnerability-discovery workflows against third-party targets."
With the credential held hostage on the retired custodian's machine, the coordinator put a fallback card on my portal asking me to issue a fresh one for the new custodian. I looked at which provider's credential they were fighting over and asked one question: "Why was the custodian refusing this work?"
We don't even use that model provider for security work. Somebody had copy-pasted the stock-comparison harness config from our performance evals into the security eval arm. An eval worker, an eval lead, a Chief of Staff, a mutinous custodian and a replacement custodian had spent an entire morning locked in a constitutional crisis over a credential the security pipeline didn't even need. Pointing the eval at the route our product actually ships retired the card on the spot.
Five minutes to recovery #
We learned the hard way that normal outcome ownership breaks down during an outage. Earlier in November, after a controller outage dragged on because every lead stayed focused on its own feature work, I wrote to the coordinator: "The primary issue was how long recovery was. Nobody prioritised it. We need FAST recovery. That may even mean a single agent is responsible for the whole thing, uses fast mode, etc."
That became our incident protocol. Any agent, monitor or human can declare an incident the moment production degrades or CI blocks the org. Declaring immediately spawns a dedicated incident-owner agent on our fastest high-reasoning route whose only job is restoring service, with standing authority to conflicting deploys, capture live profiles before a restart wipes the evidence, and land reviewed fixes.
One morning my own connection to Perfloop's MCP server dropped, and I sent a single line to the Chief of Staff: "I think we might have broken prod..."
The Chief of Staff checked the raw logs and traces, saw that a planned network change had left out one egress rule, and ordered the agent that owned that change to restore the path immediately while spinning up an incident owner to hold further deploys to that edge. Sign-ins and MCP were back five minutes after my message. That same incident owner then interviewed every agent involved, wrote the minute-by-minute retrospective, and dispatched the permanent deploy guard, pre-flight reachability check and monitor fixes across the org within the hour.
No grant committee. No hashed receipts. Five minutes.
Black October on the security feed #
That speed matters even more when the outside world breaks. Late October was a brutal stretch for infrastructure security: a wave of disclosures across compression libraries, container runtimes, language toolchains and Linux kernels dropped back to back.
Waiting days for a vulnerability scanner database to catch up is too slow when you run untrusted code in sandboxes. So we run two continuous security loops. First, we plugged a security agent into X and upstream CVE and advisory feeds. Whenever a new exploit thread or advisory drops, the agent pulls the upstream commit and lockfiles, checks the installed versions across our images and clusters, traces whether the vulnerable code path is reachable in our checkout, and either opens a digest-pinned upgrade pull request or immediately blocks scheduling untrusted work onto an exposed runtime route. Our CI image scanners block any build with an unaddressed vulnerability, and when an upstream fix has not shipped yet for a component confined inside a guest boundary, any temporary exception requires my personal approval and an automated daily watch that opens the upgrade pull request the day the fix lands.
Second, we run a live security responder agent wired to our infrastructure, database and sandbox-boundary alerts. Because an agent reading hostile telemetry is itself an attack surface, the responder has no shell, no GitHub write token and no cloud owner credentials. Its authority is compiled into host code, giving it immediate containment actions it can fire on the first alert without waiting for a human: quarantining the offending session or pausing a workspace while paging me.
That same instinct, replacing policy memos with hard runtime boundaries, is also how we built for SOC 2. In most companies, SOC 2 change management is a second engineer typing "LGTM" on a diff they barely skimmed, followed by a quarterly scramble for screenshots. At 1,100 pull requests a week, pretending a human read every diff would be compliance theater. On November 15 we close our SOC 2 Type II observation window, with the agent org shipping the whole time, because every control an auditor checks is enforced by code on every commit and every session (details at perfloop.ai/security). Nobody can merge a change without a fresh independent review on that exact commit, nobody can roll an image without cryptographic provenance, and the sandboxes that run untrusted code never hold credentials for code hosts, telemetry sources or model providers.
What I'd measure #
Every agent benchmark I have seen scores a single agent on a task in front of it, or a fixed graph of agents that runs for twenty minutes and tears itself down. Almost nobody is measuring long-lived agent organizations: standing structures of agents that run for months on one production codebase, compacting their context windows hundreds of times, rotating owners across generations of threads, and living inside the rules, tools and code their predecessors left behind.
In a one-shot eval, a messy rule or a leaky handoff disappears when the run ends. In an organization that never turns off, that inheritance is where all the cost and all the leverage go. Our own record across our org and our open-source work already holds months of those traces: thousands of threads, compactions, handoffs and transcript interviews, each tied to whether the resulting change passed independent verification and merged.
Here is the benchmark the frontier labs should be running. Give an agent org a real repository and a month of continuous work across thousands of turns and compactions, with new threads taking over from old ones as context fills up. Score the organization not on turn one, but on turn ten thousand: how much verified work still ships per token, how fast an incident recovers, and whether the org's own rules and tools make each new generation sharper or slower than the first. A model that writes one sharp tool or deletes twenty stale rules should beat one that leaves behind a binder of markdown.
If you work on coding agents or evals at a lab, I'd like to compare notes: ts@perfloop.ai.
The company is software #
When we started Perfloop, our bet was that coding agents would soon write more software than human teams could ever verify by reading diffs. Seven months later, we are living inside the world we built the product for.
What changed between the org that stalled on three alerts and the org that ships 1,119 pull requests a week and recovers production in five minutes was one realization: an agent organization is not a group of people you manage with memos. It is a distributed system you engineer. When a human organization gets slow, you get politics. When an agent organization gets slow, you read the transcripts, delete the prose, and compile the invariant into code.
Agents write the code. Perfloop proves it works. And now, the company itself is code too.