Your agent's memory is a liability: track state, not history

There is a French draft in one of my test apps that I keep like a fossil. Article 96924661: a title, a slug, zero paragraphs. A June translation run created it; I remember that run as green. I cannot check it: the n8n execution history from June is purged. What survives is a screenshot and the likeliest path, read off the canvas and never reproduced: the article is created before its paragraphs, so if the split step yields nothing, "Create FR paragraph" runs without error on nothing and the report node declares success. Green all the way down. The draft is still empty.

As I wrote in June, in the piece on automating an app with n8n and MCP, "our runs burned 40,000 to 77,000 input tokens per minute, past the rate limits of an entry-tier API account, whatever the model." The survivor was not an agent: deterministic plumbing, plus one model call per article. One line of context so you know where I stand: I run engineering at GoodBarber, an app platform, and we run a production MCP server everything below runs on systems I operate myself.

Then a preprint crossed my feed at the end of August with a headline number: 16 times fewer tokens, 94 percent once you do the subtraction, if your agent keeps state instead of history. I had lived the mechanism before the paper, so I read it back. Here is the cell the number comes from, my June job replayed five ways, what our server learned the hard way, and the rule the paper does not have.

The preprint is SKILL.state, arXiv 2608.26263, v1 posted on August 26, 2026, by Sanket Badhe and Priyanka Tiwari at Google LLC with Jonghyun Chung at Purdue. At every step the model receives an immutable skill specification, the current structured state as JSON, and the last observation; the reasoning trace is discarded once the runtime has validated its state patch. The prompt stays roughly constant, so cumulative cost is O(T) instead of O(T²).

The 94 percent is exact and it is one cell. Table 1, the authors' synthetic Warehouse environment, Gemini-3-Flash, horizon T = 100: 65,408 cumulative tokens against 1,062,387 for the stateful baseline, a 16.2x gap. Same column at T = 10: 43 percent. On the public benchmarks, InterCode CTF and τ-Bench, the reduction lands between 11 and 66 percent depending on the baseline. The gain is a horizon effect.

The result that matters is elsewhere. At equal budget, roughly 1,800 prompt tokens for every runtime at T = 100, accuracy was 0.94 for the structured state, 0.52 for a capped summary, 0.22 for ReAct plus LLMLingua, 0.18 for a truncated window. The saving is a side effect. The structure is the result.

On September 3, 2026, I counted our tool inventories with the count_tokens endpoint against claude-haiku-4-5. My n8n test app, 62 tools: 17,763 tokens. The larger test app, 77 tools: 23,174. The seven tools a translation needs: 2,416. One default page of cms_list_articles

on the small app: 16,987. The paper's mean prompt per step, all three inputs included: 1,905.

A ReAct loop on that app opens every turn with nine times the paper's whole prompt in schemas alone, and each list it calls adds nine more that history never forgets. D's six-field state object counted 56 tokens; a create-and-read-back envelope, 112.

Same job, same six English articles, same test app, September 3, 2026, claude-haiku-4-5, prepaid credits; the table is run 1, except D, the best of its six versions. Every French draft was created as status: draft

with a marker, then deleted. Verification is deterministic and outside the model: the five checks in the code block further down.

A does C's job with 160 times the input tokens. B, with 55 fewer tools, still needs 125 times: trimming the inventory does not fix a loop that re-reads its own transcript. A peaked at 694,434 prompt tokens in a 60-second window and never saw a 429; the account is not where it was in June, so I will not quote a June limit. C is the June survivor, a stdlib script now that the n8n workflow no longer runs: O(1) per article, and its prompt was the paper's triplet before anyone named it.

A second pass the same day reproduced A within 0.1 percent (1,158,626 input tokens) and B 13 percent lower (773,767); C used the same 7,135 tokens and E came out at the same $0.306.

E, arm A with cache breakpoints: 123 uncached input tokens, 73,114 written to cache, 1,080,099 read from it, 93 percent of the prompt served from cache, a quarter of A's price. Caching makes the transcript cheap. It does not make it small, or a sufficient statistic of anything.

And D, my own SKILL.state, never finished. Six runtime versions in one afternoon, 80 to 100 steps each, prompt flat between 1,400 and 2,500 tokens per step, best result 4 of 6. Each version failed somewhere new. The first schema carried ids and statuses but not the translation, so the model re-read the source at every step, 33 reads for 6 articles, and the runtime filed one French id under two sources. With the translation in state, the model invented tool arguments, author_id

, body

, language

, and the server refused 74 creates in a row. With arguments filtered against the tool schema, it re-read and re-created: 52 reads, 17 drafts for 6 sources. With the runtime owning the state machine and naming the next expected action, it created 70 paragraphs on the same drafts: after each write the runtime read the article first and the paragraph list second, so the list came from the 60-second GET cache, empty, for about 25 cycles, then 131,991 bytes at once. My commit rule was right. My read path was the cached one. With the read-back taken off the cache, 4 of 6, then a patch rejected 52 times for a string where an integer was expected. The paper reports that on small open-weight models 68 percent of failures are premature overwrites of the state; on claude-haiku-4-5 mine were the runtime's, and every one of them was a validation the runtime did not do yet. The state was constant. Sufficient is the hard part.

On September 3, 2026, I scanned my local Claude Code transcripts, counters only, no content: 264 files, 175 sessions with at least 30 assistant turns, 32,607 turns. Prompt per turn (input, cache creation and cache reads) climbs until a compaction, then drops: 37 drops of more than half, a median cost of 2.2 times a constant-size state, a median cache read share of 0.968. The longest session ran 2,444 turns and 1,071,111,255 prompt tokens, 6.89 times a constant state, 97 percent of it cache reads. That is Daniel Vaughan's point about Codex CLI: O(T²/K) by chunks, not O(T). I have never noticed a decision lost to compaction and will not claim one. I can only vouch for the bill.

Our MCP server stopped keeping history on August 10, 2026. Since then the transport is stateless: no Mcp-Session-Id

, no session cap. What the server keeps per agent, keyed by the sha256 of the access token: the last ten tool names for one hour, enough to push the read that follows a write past the cache, and since August 11 a sliding counter of 1,000 requests per minute that answers 429 with Retry-After

. Ten names and a number.

Measured on September 3, 2026, 50 cycles on a test app. A get straight after a delete: 3 stale reads out of 50, the August fix holds, not perfectly. Delete, list, get: the deleted object came back in 50 of 50 cycles and stayed a median 61.1 seconds, maximum 61.2, the 60-second GET cache doing what it was told. The bypass covers only the read right after the write; a list resets it. Three limits I read in the code rather than measured: another token never gets the bypass, nothing invalidates on write, and on an upstream failure the server serves stale cache on purpose. Read-back through a cache lies in both directions, as I wrote in the write-safety piece. It still does.

The rule I want, and the one the paper does not have: an agent's state is not what the model reports. It is what the world confirmed. Every write tool on our server returns _mcp_policy.verification_required: true

, except six fire-and-forget push tools. D's runtime turned that policy into a commit rule:

commit rule, enforced by the script, never by the model:
  1. the model proposes a patch: {"set": {...}, "delete": [...]}
  2. the runtime validates it: an unknown key, a wrong type or any touch
     of verified_fr, and the patch is rejected, state untouched
  3. an id enters verified_fr only after cms_get_article passes every check:
     exists, status draft, at least one paragraph, no residual delimiter,
     body length within 0.6 to 1.8 of the EN body
  4. a failed read-back commits nothing; the failure is the next observation
  a 200 on cms_create_article is not a commit. The read-back is.

On one article, 20 identical calls at temperature 0 on September 3 gave one distinct output; the eight offline checks I run on a translation, a different set from the five runtime checks above, caught all six faults I injected, and a status code would have caught none. Deterministic checks, not a 200.

Running that rule against our own server hit the same holes I had logged in August, and the September 3 cycles confirmed them. Ticket B: a delete returns only the policy envelope, no deleted

, no id

, no status

. Ticket C: cms_create_article

defaults to status: published

, unchanged since May 13. Ticket D: a create on September 3 returned cms_get_article

, cms_get_article_paragraph

and cms_get_event

as verification tools, the first three read tools in alphabetical order, while the tool plan points its verification call at cms_list_cms_sections

. Follow the plan to the letter and you verify an article by listing sections; follow the envelope and you may end up reading an event.

The June draft reads differently now: an invalid state patch, created without verified, accepted by a run whose only witness is my memory. IDs, not vibes.

My scheduled Claude Code tasks had the triplet before I read the paper. Each starts from a fresh context: a daily ops review that reads Nagios and BuildBot, daily code reviews of two workspaces, a Monday glossary sentinel. From local transcript counters over August 1 to September 2, 2026: the ops review ran 30 times, median 25 turns, 49 uncached input tokens and 1,529,904 cache-read tokens per run, about 14 tool calls a run. The two code reviews, 23 and 29 runs, read 14,689,395 and 10,637,775 cache tokens per median run. The sentinel, 3 runs, 820,312.

A few dozen to a few hundred uncached tokens per run, and the cached prefix is the rest of the prompt: spec, then observation. Where is the state? In Nagios, in git, in the CMS: the system of record, never the agent. The paper's architecture, state pushed one layer down, no validator anywhere.

The sentinel is the closest thing I have to a validator. A cloud routine created on August 12 ran twice, both green; I replaced it on August 17 on a design argument, not an observed drift: a routine that rewrites its script each week is a moving specification. The replacement is a frozen script with 22 negative test cases, under version control; its four reports, August 17, 25, 31 and September 2, were all green on 170 pages in 89, 139, 116 and 166 seconds. The state is the sitemap. The validator is the test file.

The paper's third limitation is the one I hit last month: when the task is the trajectory itself, throwing the history away destroys the work. My llms.txt forensic needed 121 days of logs and 1,321 requests to find 26 AI-labeled hits; the facts existed only in the sequence, and no crawler state schema would have held them. There, the log is not the memory. The log is the deliverable.

What the paper is not: synthetic-first, a strong horizon effect, a stateful baseline that pads the transcript with a state block rather than using LangGraph idiomatically, single-agent only. What our server has not shipped: any of it. No validator in front of any state file, no idempotency keys, no version-bound writes, no plan-then-commit object, no semantic span attributes, no token fingerprint column in the usage log. Tickets B, C and D were still open at our August 28 commit. D never finished.

What is cheap, and what I am doing next:

What does your prompt look like at step 50, and who decided what goes in it? If you run a scheduled agent: where does its state live between runs, what validates it before the next run reads it, and how big is it in tokens? I am genuinely curious whether anyone has a schema in front of that file, or whether we are all editing JSON by hand and hoping. I will answer with token counts where I can.