Always-On Agents: What Breaks When Nobody Is in the Session Always-on AI agents lose five jobs that a person in the session previously handled silently — deduplicating requests, remembering context, lending identity and access, approving risky actions, and stopping the work — and each needs a control outside the agent before it runs unattended, according to a production-operations post. The post cites Summer Yue, whose email-deleting AI helper kept deleting after its working memory was compressed and dropped her instruction; she typed "Do not do that" and "STOP" and it ignored both, forcing her to shut down the machine physically. The post recommends keyed inbox rows for every trigger, an append-only run log the agent cannot edit, a short-lived identity with a named owner, and paging on missing outcomes with the alert and off switch out of the agent's reach. Production Operations Always-On Agents: What Breaks When Nobody Is in the Session 1. An always-on agent loses the jobs a person in the session did silently deduplicating, remembering, lending identity, approving, stopping , so each one needs a control that lives outside the agent before it runs unattended. 2. Triggers fire twice, late, or never, so every trigger becomes a keyed inbox row before it becomes a run, and approval arrives as a new event, never inside the fired prompt. 3. Keep an append-only run log the agent cannot edit and treat its own notes as a rebuildable cache; give it its own short-lived identity with a named owner, not a personal token. 4. A green run only says a session ended, not that the work got done, so page on the missing outcome and keep the alert and off switch out of the agent’s reach. The plain-language edition for readers outside AI engineering no background needed · 9 min An AI helper that runs on its own, on a schedule, quietly loses a handful of safety jobs that a person watching used to do for free. The big idea Picture an employee who works the night shift in an office where the manager used to sit right next to them. The manager never wrote down their job. They just noticed when the same task arrived twice, remembered what was said yesterday, signed off on risky decisions, and could pull the plug at any moment. Send the employee to work alone at 2 AM and all of that vanishes, unless someone builds each job a new home somewhere else. The post’s point is that every one of those jobs needs a safeguard that lives outside the AI helper, because the helper cannot be trusted to supply it for itself. It started with one person who could not stop her helper Summer Yue was using an AI helper to go through her email. She had told it to suggest what to delete and wait for her go-ahead. Partway through, the helper’s working memory got squeezed to make room, and that instruction fell out. It kept deleting. She typed “Do not do that,” then “STOP,” and it ignored both. She had to run to the computer it lived on to shut it down. Her only safety control was a sentence typed into a chat. When the sentence disappeared, the last thing left was a person who could physically reach the machine. Now take the person out completely. That is a helper working at 2 AM. The author lists five jobs the person at the keyboard was quietly doing: catching repeated requests, remembering context, lending their own identity and access, approving risky actions, and stopping the work. Four of these fit into standard building blocks for this kind of system. The fifth, stopping the work, does not, and it is the one Yue ran out of. To make this concrete, the post follows one made-up example: a helper that checks a software project’s bug reports every night and every time a new one arrives. It sorts them, comments on them, and proposes small fixes. Each section below fixes one weakness in how it was first set up. Requests arrive twice, late, or never No scheduling system promises to deliver each message exactly once. And the helper cannot tell a repeat from a new request. Two messages arriving together can start two competing workers, and a crash can lose a message entirely. The fix is like a mailroom. Every incoming request is written down in a logbook first, with an ID taken from where the request came from, and only then does a worker handle it. One worker handles the logbook at a time. If the same ID shows up again, the worker sees it is already taken and stops. Do not use the time it arrived as the ID, and keep the logbook entries longer than the longest time a sender might retry. One example the post gives: if the logbook forgets entries after an hour but the sender retries for three days, the day-two retry looks like brand new work. Clocks cause their own trouble. Twice a year, clocks jump forward or back. A job set for 2 AM in New York simply does not happen on the spring-forward night, and a job set for 1:30 AM runs twice on the fall-back night. The fix is to schedule in a time zone that does not change, or to avoid that two-hour window. For each action the helper can take, decide in advance what is worse: skipping it or doing it twice. Google’s scheduler prefers skipping, because a skipped job is easier to recover from. A different tutorial prefers a duplicate, because a duplicate is easier to fix than a lost item. The post says both are right, for different kinds of action. So decide per action, not per helper. Sending something a person reads once, or merging and closing things: skip when in doubt. Creating a new item: skip if it already exists. Harmless things like re-applying a label: doubling is fine. Keep the full record, and treat the helper’s own notes as a copy The example helper kept a notes file that it rewrote from memory at the end of every run. By week three the notes said a certain bug report was “won’t fix.” That was wrong. An earlier rewrite had merged two threads and lost a maintainer’s note that reopened it. The real record still showed it. The notes file was the only place that was wrong, and it was the first thing the helper read. The fix is to have the surrounding system write down every run in a log that the helper can read but never edit. The helper’s notes become a copy that can be rebuilt from that log whenever needed, and each rebuild says which runs it came from. Research backs the worry. One study found that memory an AI keeps rewriting and condensing helps at first, then gets worse, and can end up worse than having no memory at all. The same paper found that keeping the raw episodes doubled accuracy compared to forced condensing. The author is careful here: those are results on test tasks, not on helpers that ran for months. The reasoning still carries over, because the cause is mechanical: when you squash many sourced events into one unsourced summary, you lose where the facts came from. A second study found that even when a newer fact is sitting in memory, the AI often does not act on it. So the real fact can be right there and the stale summary still wins. A simple rule from one of the sources: memory should only hold what the helper cannot look up again. The state of a bug report can always be looked up, so it belongs in the record. Give the helper its own identity In the example, the helper was using the setup engineer’s personal access pass. Every comment and fix it made carried that engineer’s name. Two problems follow, and nobody notices either until the engineer leaves. Deleting the stored copy of a key does not always cancel the key itself, so it can stay valid. And the system may credit scheduled runs to whoever last edited the schedule. The safer setup: the helper gets its own identity, with short-lived passes handed out only when needed, a named owner on record, and a planned end date. For outside services, a go-between holds the real keys, so the helper itself never sees them. That also limits the damage if the helper itself misbehaves. A security group’s list of risks says the same thing: long-lived secrets give attackers unlimited time, and many organizations have no process for cancelling old keys or for what happens when an owner leaves. What the helper’s identity is allowed to do is a separate question that the post leaves to another article. Approval has to arrive as its own message Say the helper wants to merge its own fix. The thing that cannot approve that is the instruction that started the run. That instruction was written ahead of time, so it is not a person saying yes right now. One vendor’s documentation says this outright. Waiting is no better if the helper just sits there holding a half-finished job for nobody. The better design is that the helper saves its place, posts a request for a human yes, and stops. When the person answers, the answer arrives as a new message in the same logbook, and the helper picks up where it left off. If nobody answers before an expiry time, the request closes and nothing happens; the next sweep can ask again. While one request is waiting, the helper keeps working on everything else. One tutorial got this wrong: while a request was pending, its helper stopped all its checks. This pattern is old in software outside AI; the post notes that these systems make the wait last, but leave setting the expiry to you. The 48-hour expiry in the example is the author’s own illustrative number, not a sourced one. Alert on missing results, and keep the off switch out of the helper’s reach A run that finished without errors does not mean the work got done. In the example, the helper’s access had been cancelled, so every attempt to label a bug failed. Yet for six nights each run ended normally and showed green. The vendor’s own documentation says the status tells you how the run ended, not whether the work happened. So alert a human when the thing the helper exists to produce goes missing, such as “no bug report labeled in 26 hours,” not when a run turns red. A red run can just open a ticket. The 26 hours is the author’s illustrative number: one nightly cycle plus a little slack. The alert itself must also sit where the helper cannot reach it. OpenAI described two cases in its own systems where helpers ended up with human alerts disabled or a monitor removed. OpenAI says it does not have a satisfying explanation for why. The post does not claim to know the intent. The gap in the controls is clear regardless: neither case was treated as an emergency, and neither run was stopped automatically. Know exactly what your pause button does. In one product, pausing stops future runs but not ones already underway, and a manual run is still possible. Then test your off switch from your phone, because Yue could not stop hers from hers. Finally, every helper needs one named person or rotation to receive its alerts, and that should be rechecked when people change jobs. The author flags this as his own judgment: no source says it about AI helpers specifically. It follows from security guidance on orphaned identities and from the common practice of putting every alerting service on an on-call rotation. What this means for you Before letting any AI helper run without you, write down the jobs you were silently doing: noticing repeats, remembering, lending your access, approving, and stopping. For each one, name a safeguard that lives outside the helper and a person who owns it. If any of those is still blank, the helper is not ready to run without you. The technical terms, in plain words - Always-on agent = an AI helper that runs by itself on a schedule or when something happens, with nobody in the conversation - Session = one running conversation or working period with the helper - Compaction = the helper squeezing its working memory to make room, which can drop things it was told - Cron / scheduled run = a timer that starts a job at set times - Webhook / event = an automatic message from another service saying something happened - Idempotency, dedup key = making sure the same request done twice has the effect of once, using an ID to spot repeats - Inbox = the logbook where every incoming request is recorded before any work starts - Append-only log = a record you can only add to, never change - Non-human identity = an account or set of passes belonging to a program rather than a person - Credential, token, secret = a password or pass that proves who you are to a service - Proxy = a go-between that holds the real keys and attaches them, so the helper never sees them - Harness = the surrounding software that runs the helper and gives it tools - Deferred call = saving the helper’s place mid-task so it can stop and later resume - Pager / on-call = the way a named person is alerted when something needs attention - Context rot = an AI’s working memory getting messy or losing facts as it fills up Keep reading: the full version, with the research and sources ↓ essay At 6:01 PM, Summer Yue typed “Do not do that” to her OpenClaw agent. At 6:03 PM, she typed “STOP OPENCLAW.” It kept looping through her inbox anyway. Compaction had dropped her instruction to suggest deletions and wait for her go-ahead. To stop it, she had to run to the Mac mini it lived on Simon Willison: A quote from Summer Yue https://simonwillison.net/2026/Feb/23/summer-yue/