You ask the agent to add a section to a site. It writes the template on the first try. The template calls a helper function, passes the right arguments, and looks like every other template in the codebase.
The build passes.
Then you run it, and you get a fatal error. Call to undefined function getCollectionBySlug().
Here is the part that wastes your afternoon: the code was plausible. A broken build is easy to catch. Plausible code pointing at an API that is not there gets past a fast read, because every individual token looks reasonable. Nothing about that template looks wrong until you run it.
The agent is not malfunctioning. It is working with what it has:
Better prompts do not fix this. You can paste the docs into the context window, write a long instructions file, and describe the correct signature. It helps a little. It does not solve it, because the docs you pasted are a copy, and copies drift from what is actually installed.
The real problem is the source of truth. The agent is guessing about a system that knows exactly what it is.
Your installed app already knows its own schemas, its own routes, and which functions this version actually exposes. Nothing was asking it.
Ship a small server inside the app that answers three questions. The protocol does not matter for this post; a plain HTTP endpoint works fine. What matters is that it is served from the running install, so it cannot drift from reality.
1. Schemas. The real collections and fields on this site, not the generic example from the docs.
2. Live content. Real records, so the agent can look at what exists before it changes anything. This one prevents a whole category of mistake: inventing plausible field values for a record that was never queried.
3. Documentation for this version. The docs ship with the install and are served from it, so an agent working on 3.6 reads 3.6's functions and field types. There is no gap between what the docs say and what the site does.
The third one did the most work in my case. Once the agent could look up a real signature instead of recalling one, it stopped guessing. The reason is obvious in hindsight: your installed version is authoritative about your installed version, and a website is not.
There is a second benefit that took me longer to notice. Customers do not always upgrade the moment you release. An agent that reads documentation from a marketing site will confidently use the feature you shipped last week, on the version they are still running. Documentation served from the install describes exactly what that site can do.
Reading is the easy half. Letting an agent write is where people are right to be nervous.
The rule I settled on: the agent gets no privileged path. Every write goes through the same validation and fires the same events as a save from the admin interface. Permissions come from the same access groups that govern human editors. A token issued to the agent gets exactly the checks a signed-in user gets.
The benefit is that "can the agent do this?" gets answered in one place, and that place already exists. You are not maintaining two authorization systems. You are not maintaining one authorization system and one audit trail that nobody reads.
If your answer to "can the agent touch this?" lives somewhere other than your existing permission code, that is the actual bug. The fabricated function was a symptom.
Here is the shape of it, stripped down. The framework does not matter; the three questions do.
Suppose your CMS has a products collection. Each product has a handle, a title, and an optional discount that is either null or a number between 0 and 100. On version 3.6 the accessor is getCollectionBySlug(). On 3.5 it was findCollection(). The rename happened two releases ago.
An agent asked to render a discount banner will pick one. Which one it picks depends on what it has seen most often, and two releases apart is close enough that both appear in its training data. Nothing about the task signals that the version matters, so the agent has no reason to prefer the right one.
Now give it a way to ask, and the version question answers itself:
GET /__self/versions
-> { "app": "acme-cms", "version": "3.6.4" }
GET /__self/schemas/products
-> { "handle": "string", "title": "string", "discount": "number or null, 0 to 100" }
GET /__self/docs/3.6/functions
-> { "getCollectionBySlug": { "params": ["slug"], "returns": "Collection" } }
The agent now has the right name, the right shape, and the constraint it would otherwise have invented. Nobody had to teach it the rename. The install already knew.
The discount line matters more than it looks. An agent that guesses the field will happily render undefined% and hand you a bug report about the template. An agent that reads the nullable type will write the guard clause, because the type told it the guard is required.
Reading is safe in a way writing is not, and it is worth being precise about why.
A read cannot change state. The worst outcome of a wrong read is a wrong answer that a human immediately sees and corrects. Writes invert that: the wrong write looks exactly like the right one until the data is already changed, and by then the original value is gone.
That asymmetry is the whole argument for keeping reads and writes on the same door. If the agent reads schemas through the install but writes through some convenience layer you added, the two paths drift. The read path is verified against reality on every call. The write path is verified against your assumptions, once, when you wrote it.
Worth being clear about, because these posts usually skip it.
If your agent is working from documentation rather than from the running system, it is guessing with good manners. Make it ask the install instead.
That single change removed the fabricated-API class of bug from my work entirely, and it took an afternoon. The part that took longer was accepting that the fix belonged in the app, not in the prompt.