{"slug": "reducing-grok-bot-consumption-with-durable-state", "title": "Reducing Grok Bot consumption with durable state", "summary": "Automaton's durable-state redesign cut GrokBot inference calls by 95%, with 19 of 20 turns served from a local SQLite store instead of replaying model context. The architecture treats context as a cache, persisting claims, job artifacts, and receipts so repeated work returns stored results without an OpenRouter request. Puppetmaster remains the job runtime, and the next hardening step extends reuse gates with repository, profile, input, and freshness fingerprints.", "body_md": "On this repeated-work replay, 19/20 turns avoided inference (95%). That number comes from changing the architecture, not from discounting a single new task.\n\nGrokBot should treat context as a cache, not as its database.\n\nThe expensive design is a permanent head seat that carries permanent role runtimes, internal transcripts, and self-correction history. Each new turn replays more context, and the system pays again for work that another worker already completed.\n\nThe solution is to move memory and completed work out of the model context and into durable state.\n\nDo not keep every agent alive as a transcript-carrying participant. A mouth handles a turn, then the runtime can disappear. The durable backend keeps the things that have future value:\n\n- session snapshots and thread state\n- claims: verified spoken lines and findings\n- task results and artifact references\n- agent ownership and job provenance\n- per-turn usage receipts\n\nIn Automaton, this is a local SQLite store. A completed job becomes a job-sourced claim instead of remaining only inside a worker transcript. Inserting the same claim again is idempotent.\n\nThe request path is deliberately ordered:\n\n```\nuser turn\n    |\n    v\nowner-scoped claim lookup\n    |                         \\\n    | hit                      \\ miss\n    v                           v\nreturn stored result       build bounded working set\nwrite zero-call receipt    call the mouth once\n                                |\n                                v\n                         persist result and receipt\n```\n\nOn a relevant claim hit, the system returns the stored result without making\nan OpenRouter request. The receipt records `outcome=hit`\n\nand\n`inferenceAvoided=true`\n\n. This is a real zero-call cache hit, not a prompt that\nasks a model to remember something.\n\nOnly a miss reaches inference. The miss prompt contains:\n\n- the system instruction\n- relevant claims retrieved from durable state\n- the selected story, when the compaction layer has one\n- a small recent-message tail\n\nIt does not carry the entire historical transcript. The shipped Automaton mouth used a recent tail of eight messages.\n\nThe same rule applies to jobs. A previous artifact is reusable only when its identity and freshness match the new request:\n\n- repository and profile\n- owning agent\n- normalized task\n- relevant inputs\n- successful, live terminal status\n- substantive artifact or finding\n\nAutomaton's current analyze-reuse gate enforces the owning agent, normalized goal, live successful status, and substantive-artifact checks. Extending the identity with repository, profile, input, and freshness fingerprints is the next hardening step.\n\nPuppetmaster remains the job runtime, not the ordinary-chat runtime. Job handles are persisted so a relaunch attaches to an existing job instead of spawning a duplicate. Analyze launches carry a local launch key for idempotent recovery. Implement jobs always get fresh sandboxes and are never reused as prior implementation work.\n\nAn attached job with an unavailable status is bounded and fails closed. It is not watched forever and it is not reported as successful merely because the worker disappeared.\n\nOlder conversation should not remain an ever-growing prompt source. The retention layer uses the catalog-residual shape:\n\n- extractive handles for older material\n- a bounded, last-wins selected story\n- a session-scoped SQLite FTS vault for lexical retrieval\n\nThis layer is derived context. It is not the authority. Claims and verified job artifacts remain authoritative. FTS hits can help retrieve context but cannot answer a query directly, and the full vault is never injected into every prompt.\n\nIn the current Automaton checkpoint, the durable claims, bounded working set, and reuse ledger are shipped; catalog-residual compaction is the next retention boundary. The separation is intentional: compaction can change prompt context without changing the truth of a completed job.\n\nEvery turn writes a receipt with:\n\n- hit or miss outcome\n- model and terminal status\n- nullable prompt tokens, completion tokens, and cost\n- whether inference was avoided\n- whether inference was attempted\n\nThe ledger counts avoided calls and attempted calls separately. Provider usage that is missing remains unknown; it is not converted into fake zero cost. This makes the claimed reduction testable:\n\n```\navoided calls / total turns\nknown token totals\nknown provider cost\nunknown usage that still needs attribution\n```\n\nA missing API key is not an inference attempt. A failed provider call is an attempt. Those cases must not be collapsed into one misleading number.\n\nThe important boundaries in Automaton are:\n\n`StaffStore`\n\n: durable sessions, claims, receipts, and aggregate metrics`queryFirst`\n\n: authoritative claim lookup before inference`buildWorkingSet`\n\n: system prompt plus recalled claims plus bounded tail`ensureMouth`\n\n: hit handling, one bounded inference path, and receipts`ensureDispatched`\n\n: persisted attachment and safe analyze reuse`Puppetmaster`\n\nadapters: sandbox jobs, artifact references, and status\n\nThe product UI is not the cache. The chat transcript is not the artifact store. Puppetmaster is not the mouth. Each layer has one job.\n\nThe high-cost system pays for coordination every time:\n\n``` php\nquestion -> permanent roles -> growing transcripts -> repeated inference\n```\n\nThe durable system pays for novel work once:\n\n``` php\nquestion -> durable lookup -> hit -> answer\n                         \\\n                          miss -> small prompt -> work -> durable result\n```\n\nAfter one worker has completed a reusable result, the next matching request is a query. It does not need another head-agent debate, another permanent role runtime, or another full transcript.\n\nThe 95% figure is a measured session-level result on one repeated-work\nreplay, not a claim that every workload or every Grok Bot user sees 95%. The\nreceipt ledger is how to verify the number. See [Measured repeated-work replay](#measured-repeated-work-replay).\n\nOn this repeated-work replay, 19/20 turns avoided inference (95%).\n\nThis is one paid miss plus 19 zero-call recalls of a stored job finding. Turn 1\nasks a novel question that is not a recall; `queryFirst`\n\nmisses and one mocked\nmouth call is the paid inference. Chat misses do not `remember()`\n\nthemselves.\nAfter that miss the replay seeds one job-sourced Kernel claim (`The ledger replay is deterministic.`\n\n) as if a worker had finished. Turns 2–20 ask `what did Kernel find about ledger replay`\n\nand hit with `inferenceAvoided=true`\n\nand\nno further ChatFn calls.\n\nThis is not Cary Palmer's live mixed ledger. That mix was 1 hit / 51 turns, and it is not this workload.\n\nThis 95% is a session-level hit rate across a day of work, not a discount on a single new task. The first look at a repo, paper, or bug still pays a full mouth call. Later turns that come back to that same finding query the store and skip the model. A typical day is mostly those later turns; that mix is why 19 of 20 turns avoided inference. One novel task is still one paid call (100% of that turn). Do not read this as \"per task 95% off\" or as \"Grok Bot users always save 95%.\"\n\nThe captured ledger is [ repeated-work-ledger.json](/professorpalmer/automaton-durable-state/blob/dev/repeated-work-ledger.json).\nRe-run from the Automaton checkout:\n\n```\nbun scripts/replay-repeated-work.ts\n```\n\nThat script uses a temp sqlite path. It does not read `~/.automaton/staff.sqlite`\n\n.\n\nTreat context as a cache, not as the database. Make agent roles ephemeral. Persist sessions, claims, task results, artifacts, provenance, and usage receipts. Query verified durable state before inference. On a miss, provide only relevant claims, the selected story, and a recent tail. Reuse work only when repository, profile, task, inputs, and freshness match. Compact old history into extractive handles, a last-wins story, and a bounded searchable vault. Pay once for work, query it next time, and measure avoided calls, tokens, and dollars honestly.\n\nThis is a backend and ownership change, not a summarization feature.", "url": "https://wpnews.pro/news/reducing-grok-bot-consumption-with-durable-state", "canonical_source": "https://github.com/professorpalmer/automaton-durable-state", "published_at": "2026-08-27 14:31:53+00:00", "updated_at": "2026-08-27 14:49:11.219514+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-products"], "entities": ["Automaton", "GrokBot", "OpenRouter", "Puppetmaster", "SQLite"], "alternates": {"html": "https://wpnews.pro/news/reducing-grok-bot-consumption-with-durable-state", "markdown": "https://wpnews.pro/news/reducing-grok-bot-consumption-with-durable-state.md", "text": "https://wpnews.pro/news/reducing-grok-bot-consumption-with-durable-state.txt", "jsonld": "https://wpnews.pro/news/reducing-grok-bot-consumption-with-durable-state.jsonld"}}