How non-engineers ship their own AI tools to production from a chat — and the one zone the engineer kept. Part 2 of A Prompt Is a Wish. A Tool Is a Law.
I opened Claude, connected to our internal platform — a single MCP server through which Claude sees every AI tool the company has — and typed what a marketer would type:
Every Monday morning, take last week's App Store and Google Play reviews, group the complaints, and post the top 5 to the team channel.
Not a word about APIs, cron or tokens. A plain work request.
The first thing the platform did was not write code. It went looking for what already existed. And it found it: fetching reviews from both stores already existed, and so did filtering by date and rating. A colleague had built them earlier, for a different task. Half the tool was done before I finished the sentence.
Then I did what anyone in a hurry does: asked for the plan right away, no questions. The platform refused — the clarifying-questions step (run_bmad, based on the open BMAD Method: brainstorming, elicitation, pre-mortem) hadn't happened yet:
⛔ Cannot confirm plan — missing steps:
- run_bmad (discovery or elicitation — ask user clarifying questions)
Complete these steps first.
This is not a polite line in a prompt that the model can skip. It's a tool that physically won't accept a plan until the questions step is done. To be honest: the gate guarantees the step happened, not that the questions were good. But "skip it" stops being an option. How a gate like this works in code — the phase lives in server-side storage and every tool checks it — I showed in Part 1.
And the questions weren't technical. The platform proposed a pre-mortem — "a month from now nobody reads the digest. Why?" — and out came: what to post in a week with almost no negative reviews; whether to group complaints across ten languages together; and what to do when the same complaint sits in the top 5 every week and turns into noise. Those are product questions, not engineering ones.
After the answers, a plan appeared — what gets reused, what gets created, where every byte of data flows — and one line:
✅ Plan registered. Now STOP and show the plan to the user.
Do NOT generate code until the user says yes.
Not a single line of code has been written up to this point. Next comes "yes", and Builder — the part of the platform that walks the author through the pipeline — writes the code, runs it through three independent reviews (adversarial, edge cases, code quality), opens a pull request, waits for the review bot and fixes its comments on its own.
And then it does something that, until recently, only a developer could do here. It merges. Right there in the same chat.
We didn't remove the engineer from the process. We narrowed them down to one zone — the platform core. Everything else, from the first sentence to production, is done by the person who came with the problem. A vibe coder, in this article, is someone who builds software by talking to an AI and doesn't read the code that comes out.
One paragraph of context, then no more theory. Everything on the platform is one of three types. A skill is pure logic, with no network and no secrets. An action is the only thing that reaches the outside world and holds keys. An agent composes skills and actions into a workflow. The author never holds a token: secrets are injected on the server.
For a long time the pipeline broke off at the last step. Everything up to the pull request was automated, and then the author went to a channel and asked someone with permissions to press Merge. Sometimes they waited for days.
When we looked at those merges closely, the picture wasn't pretty. Most Builder PRs went in without a single standing human approval. The person pressing Merge wasn't reading the diff — they were clearing a queue. There was no review left there. Only delay.
So now the author merges. But not everything.
The line isn't between "content" and "infrastructure". It's between what the author wrote and owns, and the rules everyone works under. Your file — you merge it. The file that decides the rules — you don't. Here's what that line looks like in code:
// what a Builder run may merge on its own
const SELF_MERGE_ALLOWED_PREFIXES = ["_shared/", "docs/", "test/"]; // skills, actions, agents, docs, tests
const SELF_MERGE_ALLOWED_FILES = [
"skill-runner/src/registry.ts", // a new tool must register itself
"agent-runner/src/registry.ts",
// + one generated file, written by the server, not the model
];
// explicit denials — even inside an allowed folder
const SELF_MERGE_DENIED_FILES = [/* the module that checks approval and CI */];
// path comes from the PR file list GitHub returns: repo-relative, no "..".
// If you take paths from anywhere else, normalize first — or startsWith can be bypassed.
export function isSelfMergeable(path: string): boolean {
if (SELF_MERGE_DENIED_FILES.includes(path)) return false;
if (SELF_MERGE_ALLOWED_FILES.includes(path)) return true;
return SELF_MERGE_ALLOWED_PREFIXES.some((prefix) => path.startsWith(prefix));
}
Your own skill, action, agent — you merge it yourself. The gateway, the runners (the services that execute skills and actions), CI — everything that defines how the platform works for everyone — only through a developer. If the diff contains even one file outside the zone, the merge refuses and names that file.
Registries are executable code too, and they're in the self-merge zone. So the platform doesn't trust the model's version of a registry: when the PR is created, the server rebuilds the registry from main and only appends the new entries. The model can propose a registration line, but it can't rewrite the file.
A vibe coder can ship their tool to production without an engineer. A vibe coder can't change the core without one.
Four decisions inside this gate worth stealing:
1. Merge is unavailable until the whole pipeline has passed. The merge tool isn't handed out "at the end of the conversation". It checks that every step is done: questions, plan, review, three independent reviews, final check, an open PR, green CI and the review bot's approval. Missing one — refusal, naming the missing step. Same as the plan at the top of this article: you can ask, you can't get.
2. The checker is outside the zone it checks. The module that reads approval and CI status lives inside an allowed folder — but sits on the explicit deny list. The code that decides whether a merge is allowed is never merged through self-merge. Any change to it goes through a developer.
3. You can't name someone else's PR. The merge tool doesn't take a PR as input at all. It takes the current Builder run and merges only the PR that run opened — and only if it's called by the same authenticated person who started the run. With no known identity, it refuses outright. There's nothing to merge someone else's work with: the parameter doesn't exist.
4. Not GitHub's built-in auto-merge. The obvious solution is to enable auto-merge when the PR opens. We looked at the timings of a real PR: it went green about 90 seconds after opening, and the fix for the review comments landed a few minutes later. With auto-merge, the version before the fix would have shipped — exactly in the case where the fix mattered. So the merge is pinned to the specific commit that passed the checks: if anything lands after that, the merge refuses.
Before merging, the author sees a preview in the chat: which files, how many lines and which services will go to production. Because a merge to main is the release here — there's no separate "deploy" button.
We didn't invent this line at a whiteboard. We tested it against history: we took all 296 PRs Builder had ever opened and counted how many each version of the rule would have let through to self-merge.
The first version was strict and "clean": only the folders with tools, docs and tests. It would have let through only 20% of PRs. The reason turned out to be boring: every new tool has to register itself — append a line to a shared list — and that list lived outside the allowed folders. Almost every PR touched it and fell back to a manual merge.
We added the list itself to the allowed set — as described above, the model can't hurt it, the server rebuilds it. And the share went up to 81%. The remaining 19% are changes outside the author's zone — the core, runners, CI — and they still go through a developer.
A gate that blocks almost everything isn't safe. It's useless, and people start routing around it.
An honest caveat: this is a backtest on PRs that were merged under the old rules, not a proof for the future. Self-merge itself has been live since late September — that's a first data point, and we'll recount when there's volume. But the review pipeline underneath it isn't new: 109 PRs since late June, and not one unplanned revert (why — in the next section). A hundred PRs, not thousands; a direction, not statistics.
Let me say it plainly: when you self-merge your own skill, there's no human in the review. Instead there are two layers, and they're separated on purpose.
The first layer is on the client, in the same place the code was generated: self-check plus Builder's three reviews. We've tested this layer on ourselves and we know its weakness: the code and the review live in the same context, and a model that just wrote the code is happy to find it good. Problems slipped through there.
The second layer is a review bot with a clean context. It hasn't seen the conversation and doesn't remember how the model talked itself into "this is fine". But it does know what the author wanted — from the ADR. An ADR is a short decision record, mandatory for every new tool: what the author asked for, what was decided, what data the tool touches and where it sends it. The bot reads it and checks it against the code: does the code do what's written, and does it do anything that isn't.
The difference matters. The bot gets the author's intent not from a chat, where it can be blurred and renegotiated, but from a document that stays in the repository after the conversation is over. The code and the ADR are two records of the same intent, and the bot checks that they match.
That second layer is what made the difference. Since late June, 109 PRs have been merged through the pipeline, 73 of them opened by Builder. There's been one revert in that time — a planned rollback of a test PR we used to check self-merge itself. What did slip through — imprecisions in code, a vibe coder's wording — the author fixed in the next PR after actually running their skill. Once, a more serious regression got through: entries disappeared from the registry and some tools became invisible. That was also closed with the next PR, and the registry protection described above closes exactly that class of bug. If a bug does slip through, its blast radius is one zone of one author, and it's rolled back with a revert PR through the same pipeline.
A review in the same context the code was written in is a wish. A review with a clean context is a check.
The bot has its own prompt for this repository. An excerpt:
Review like a cynical, senior reviewer with zero patience for sloppy work.
Assume problems exist. Surface what is MISSING, not just what is wrong.
Three lenses: adversarial, edge-case hunting, code quality.
CRITICAL: no network calls in skills/agents/crons (fetch or any HTTP client) ·
registry entries only added, never removed or renamed · ADR for every new
artifact · no hardcoded secrets · new secrets only through the secrets manager ·
access groups must be real groups
HIGH: platform logger, not console.log · reuse shared helpers, don't duplicate
The same three lenses as the client-side review — but with a clean context and a hard list of what blocks a merge. Before reviewing, the bot reads the repository's CLAUDE.md — the same rules Builder writes code by. So it's one rule and three enforcers: Builder writes by it, the submission tool checks it mechanically, and the bot checks it again without seeing the conversation.
The bot itself is a separate service, and its prompt lives there, not in the platform's repository. You can't change the review rules with a Builder PR.
And an important caveat: the bot is not a security boundary. It catches mistakes, not malice. An author who wants to cause harm on purpose can try to talk the reviewer around with a comment in the code — prompt injection against a reviewer is very much a thing.
The gates in this article protect against accidents: a model that cut a corner, a vibe coder who didn't know where not to go. Against malice inside the team, something else protects us: every author is an employee under their own account, every change is tied to them in the PR and the ADR, every call is in the logs. That's not "impossible". That's "not anonymous, and reversible". For an internal platform, that's an honest level. For a platform open to the outside world, it wouldn't be enough.
We don't re-review the bot's reviews by hand. Not out of laziness — there's no point. The platform's code isn't written by people: vibe coders build tools through Builder, and so do developers — we build our own skills the same way. This code is read first by the next model, not by a human.
So human readability is secondary here, and predictability isn't. Strict naming with a domain prefix, one file per artifact in its domain folder, an ADR for every tool, a registry that's only ever appended to. That isn't bureaucracy — it's navigation for a model. It's exactly why, at the start of this article, Builder immediately found the existing review fetchers: they were where they should be, named the way they should be.
A human needs readability. A model needs predictability.
Three examples — and in each one you can see who could have broken something and what stops it:
Also, briefly: scoring ad scripts against an internal storytelling framework, a funnel screen generator that follows the design system, an HR bot with routing rules in a separate skill, an analytics reference for "which column is the real revenue". None of the authors ever held a single token.
The dashboard that wasn't the problem. A marketing analyst opened Claude and built herself a dashboard. No developers. Building it in BI would have been overkill; a local HTML file solved the task. The problem was elsewhere: how to share it. We packaged it as a skill — and it became available to everyone who needed it.
She could build the tool herself. The last mile isn't code. It's distribution.
To build — about two weeks of one engineer plus infrastructure setup from DevOps. Not a dedicated platform team, not a quarter.
To run — a monthly report from mid-summer: $5 a month for the infrastructure of the whole platform (model tokens are a separate line — you pay them wherever the code runs), almost 80,000 requests (about 2,600 a day), 289 seconds of pure CPU for the month, and 30+ people who ran tools by hand. It's cheap not because Cloudflare Workers are cheap, but because a tool spends almost all its time waiting on the network — the model, an API, storage — and waiting doesn't count as CPU.
Each step below removed one more reason an author needed a developer.
Actions. At first the platform could only think. With actions it got hands: skill — brain, action — hands, agent — manager. Automations stopped needing a person in the chat: an event in the tracker → a webhook → the action calls the model on the server → the result goes back into the ticket.
Two-pass review with auto-fix. Self-check before the pull request, the bot after, and Builder fixes its comments in the same PR. That removed the question authors used to get stuck on most: "what do I do with this review comment?"
Schedules as a primitive. Cron became the same kind of type as a skill or an action: one file, same review. The ceiling is about 15 minutes per job. Enough for alerts and digests, not enough for payments.
Agents that propose fixes to themselves. If an agent keeps getting corrected the same way, it can propose a patch to itself. Off by default. The fixes go through the same PR and the same review as a new tool — no silent self-modification. And it can only touch what belongs to that agent and nobody else:
// an agent may propose a fix only to the skills it alone uses
const editable = agent.skills.filter((skill) => consumersOf(skill).length === 1);
One line closes the main risk of self-learning: a fix for one agent won't break a dozen others that reuse the same skill.
Composition instead of new agents. In early July there were 43 skills and 27 agents. Now there are 125 skills, 26 agents and 75 actions. Three times the skills, the same number of agents. New things are assembled from existing pieces more often than written from scratch.
A few rules we didn't plan. Each one came from a real failure, and each one is now enforced by the pipeline rather than remembered.
State doesn't live in the conversation. An early nine-step agent kept intermediate results — a video URL, a new card id, a file name — in the chat's working memory. In long chats it lost them. Now every data-gathering step has its own raw input field on the agent, the agent is called once with everything collected, and a workflow that waits on an external job is split into a prepare agent and a finalize agent that pass an opaque context blob between them. The submission tool blocks the old shape.
Long work is submit, poll, finalize. A webhook-triggered action gets about 30 seconds before the platform kills it — silently, with no error reaching the caller. So renders, dubbing and exports are split into separate calls, and the caller drives the loop between them. Never sleep-poll for minutes inside one invocation.
Media goes by link, not by bytes. A generated image is written to a scratch bucket and the model gets a presigned URL instead of the file. That saves around 98% of the tokens. The link lives four hours, the object is deleted after a day.
An unknown field is an error, not silence. A caller once guessed plausible field names instead of copying the schema; the tool silently ignored them and a whole script disappeared. Now a validator throws on unknown input instead of dropping it.
Agents don't call agents. Only Claude chains one agent's output into another. An agent orchestrates skills and nothing else — which keeps every hop visible in the conversation instead of buried in code.
When a tool costs one conversation, people build them "just to try". One-off skills, second versions with forgotten first ones. The catalog grows faster than it's used — and starts getting in the way of finding what's alive, for people and for the model choosing a tool.
The answer is an audit, and it needs one thing you should do before you need it: every call writes one line of structured log — name, type, user, status, duration. The audit itself is a diff:
const used = new Set(await calledToolNames({ window: "30d" })); // from logs
const registry = await listRegistry(); // everything published
const unused = registry.filter((t) => !used.has(t.name)); // nobody called it
const orphans = [...used].filter((n) => !registry.some((t) => t.name === n)); // in logs, gone from the registry
Three things we stepped on while building it:
Right now the audit lives with me locally. The next step is to make it part of the platform: a regular report on what nobody calls.
If you were an author on this platform, this is the whole path:
Access is public by default. A tool is restricted to a group only when the author explicitly asks for it — and only to a real group; Builder won't guess one from context.
The questions any manager asks before letting their team in:
Where this won't work: if you don't have one shared repository and a CI you can trust, there's nothing to build self-merge on. Start there.
I built the platform so that any AI tool one person makes would be available to everyone. For a long time that came with a caveat: anyone could build a tool, but only a developer could get it to production. Now that caveat is gone too: the author takes their own tool all the way to production.
In Part 1 I wrote that a prompt is a wish and a tool is a law. Here's the other half of the same idea. The laws — the gates, the zone boundary, the review rules — are written by an engineer, once, in the core. Tools under those laws are written by anyone.
The stricter the laws, the more people you can let in to write tools.
If you've let people without an engineering background build tools — what turned out to be their real blocker: building it, or sharing it?