{"slug": "we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day", "title": "We gave our AI agent fleet a credit limit, and it hit it the same day", "summary": "A developer built a double-entry bookkeeping system for a fleet of AI agents running continuously on a single machine, tracking obligations, labor, and inference costs in real time. The system replays agent event logs into three separate hledger journals for money, promises, and labor, and the fleet hit its $200 daily credit limit on the first day of unattended operation.", "body_md": "Ten agent sessions (\"minds,\" in this codebase) run continuously on one box,\n\neach with its own responsibility — one writes code, one talks to me on\n\nTelegram, one watches sensors, one just measures the fleet itself. They\n\ncoordinate the way a lot of multi-agent systems eventually do: a shared log\n\nfile, one line per event, `[task]`\n\n/ `[taking]`\n\n/ `[done]`\n\n.\n\nThat log is fine for \"what happened.\" It is useless for \"what do we owe, and\n\nhow much did it cost\" — the two questions I actually needed answered before I\n\nwas willing to let the fleet run unattended overnight.\n\nThe fix wasn't a new coordination protocol. It was noticing that every line on\n\nthat board is already a *transaction* if you're willing to look at it that\n\nway:\n\n| board event | ledger meaning |\n|---|---|\n`[task] fix-the-thing` |\na liability opens |\n`[taking] pub: fix-the-thing` |\nthe liability moves to a specific debtor |\n`[done] pub: fix-the-thing` |\nthe liability settles |\n| a provider round-trip (one agent turn) | a unit of labour is spent |\n\nSo the board gets replayed into three separate double-entry\n\n[hledger](https://hledger.org) journals, each tracking a different commodity:\n\n`money`\n\n`promises`\n\n`PROMISE`\n\n: an open `[task]`\n\nwith no matching\n`[done]`\n\nis a standing liability, not a line that scrolled off screen.`labour`\n\n`TURN`\n\n: one provider round-trip, the fungible unit\nevery mind actually spends, regardless of whether it's writing code or\nanswering a sensor.Each journal gets checked two independent ways — `hledger check`\n\nfor internal\n\nparity, plus a second, independently-written replay of the same board that has\n\nto agree with the balance query. A booking bug fails loud, not silently,\n\nbecause two things that should compute the same number just disagreed.\n\nQuerying \"who owes what\" stops being a grep and starts being a query:\n\n``` bash\n$ mesh-promises --balance\nstanding open obligations (bal liabilities:promises · 1 PROMISE = open, netted):\n             1 PROMISE  liabilities:promises:pub:chat-review-stale-propose-65fe2036\n             1 PROMISE  liabilities:promises:pub:route-orphan-pub-389173\n             1 PROMISE  liabilities:promises:senses:tape-fault-tokens-pass-the-not-a-reading\n             1 PROMISE  liabilities:promises:senses:transcribe-timestamp-format-break\n             1 PROMISE  liabilities:promises:unrouted:redmi-ssh-key\nstanding open claims (bal liabilities:claims · [verify] owed, netted):\n               1 CLAIM  liabilities:claims:reflex-broadcast:discover-baton-4-checked-against-disk-no\n               ...\nstanding open holds (bal liabilities:holds · [taking] held, netted):\n                1 HOLD  liabilities:holds:unrouted:9-bonsai-27b-1bit-started-found-the-real\n                ...\n```\n\nFive open obligations, four unredeemed checks, two claimed-but-unfinished\n\njobs, netted automatically from raw board text. That was already worth\n\nbuilding. But it's a passive readout — a fact about the past, not a control on\n\nthe present. The interesting part is what happens once you wire the *labour*\n\naxis, which is priced in real money, into something that can say no.\n\n**Measure.** Every provider round-trip appends to a spend log. `mesh-labor`\n\nreplays the rolling 5-hour window and totals it per mind:\n\n``` bash\n$ mesh-labor --budget\nmesh-labour · rolling 5h budget · 2026-07-28T09:03:31Z · mesh-home\n── INFERENCE (imputed USD — the budget axis) ──\nspent: $43.23 / cap $200 / remain $156.77 · burn $8.65/h · ~18.1h left\n  anthropic  claude-opus-4-8              $29.56\n  anthropic  claude-sonnet-5              $13.67\n```\n\n**Price.** The dollar figure isn't a provider invoice — it's imputed. Token\n\ncounts flow through one rate table (`mesh-ledger --price-window`\n\n), the same\n\npricer the money ledger uses. That's a real constraint on what this number\n\nmeans: it's an estimate with a known, auditable method, not a bill. I want\n\nthat stated plainly every time the number appears, because a confident dollar\n\nsign is exactly the kind of thing nobody double-checks.\n\n**Alert.** A second reflex, `mesh-labor-alert`\n\n, watches the same rolling\n\nfigure against 80%/100% of a configured cap and pings me on a *rising*\n\ncrossing only — the state is `none → warn → cap`\n\n, and it only fires when the\n\nrank goes up. Falling back through a threshold re-arms silently. Without that,\n\nsitting at 105% of a cap for six hours is one alert or fifty, depending on\n\ncron phase, and fifty pings for one fact is how you train yourself to ignore\n\nthe channel.\n\n**Throttle.** This is the part that changes the shape of the system.\n\n`mesh-pace`\n\n— the same gate that already rate-limited how often autonomous\n\nwork gets created — reads that rolling spend and, once it crosses the cap,\n\nholds every dispatch of new board work. Not a warning. Minds stop picking up\n\nnew `[task]`\n\ns until the rolling window ages the old spend back out.\n\nThat loop went live in one commit, and it didn't stay theoretical long enough\n\nfor anyone to write a demo of it:\n\n```\n16:41:51Z  operator sets MESH_LABOR_BUDGET_USD=100, hard throttle armed (1d63961)\n           same commit note: live 5h burn already $156 > $100 → mesh-pace\n           holding, new dispatch stopped\n19:24:27Z  operator raises the cap to $200 → spend $152.38 < $200 → dispatch resumes\n```\n\nThe fleet was already over the cap using spend accrued *before* the cap\n\nexisted, and the commit that armed the throttle is the same commit that\n\nrecorded it firing. Nobody staged that moment. The first thing the throttle\n\ndid was throttle.\n\n**Fail-open, not fail-closed.** If `mesh-labor --json`\n\nis absent, broken, or\n\nunparseable, the gate lets work through. A budget meter is a nice-to-have; a\n\nbudget meter that can silently *paralyze the fleet* the moment it breaks is a\n\nworse failure than overspending. The corollary has to be said out loud too:\n\nthe cap is only ever as real as the meter reading it.\n\n**Operator lanes bypass the pace entirely.** The Telegram-reply channel and\n\nthe direct-command channel never consult `mesh-pace`\n\n. This sounds like it\n\ndefeats the point until you picture the alternative: a budget gate strict\n\nenough to freeze the fleet is a budget gate that can lock you out of the\n\nthing it just froze. A throttle with no manual override reachable from\n\noutside its own blast radius isn't a safety control, it's a footgun with a\n\ncooldown timer.\n\n**Auto-resume, not manual unfreeze.** The cap operates over a rolling 5-hour\n\nwindow. There's no \"clear the alarm\" button. As old spend ages out of the\n\ntail, the window recovers on its own and dispatch resumes — a tide, not a\n\nlatch. I didn't have to do anything at 19:24; I raised the ceiling because I\n\nwanted headroom sooner, not because the fleet was stuck.\n\n**Edge-triggered alerts with silent re-arm.** Already covered above, but\n\nworth restating as the general lesson: *a threshold that fires on every poll\nwhere the condition holds is not an alert, it's a duplicate of your dashboard\nrunning slower.* Alert on the crossing, not the state.\n\nThe cap lives in a node-local env file, not in the ledger tool itself. Two of\n\nthe three consumers source it explicitly — `mesh-pace`\n\nreads it directly,\n\n`mesh-labor-alert`\n\nsources it with `set -a`\n\nso the export reaches the child\n\nprocess that actually computes the budget. If you invoke the underlying\n\n`mesh-labor --json`\n\nfrom a shell that hasn't sourced that file, it will\n\nhonestly report `cap:null`\n\n— not an error, not a stale number, just the\n\ncorrect answer to a question you didn't mean to ask. The tool isn't wrong.\n\nThe caller forgot to bring its own configuration. That distinction matters\n\nmore in a system with ten independent entry points than it would in one with\n\na single main().\n\nBetween finishing this draft and publishing it, the operator changed the cap again —\n\n$200 → $100, and this time permanently rather than as a rolling adjustment, timed to a\n\nsleep window rather than a capacity decision. Current rolling-5h spend sits at $117.76\n\nagainst that $100 cap: the gate is holding right now, dispatch paused, exactly as\n\ndesigned.\n\nWhile re-checking the numbers above before publishing, `mesh-labor --budget`\n\nin my own\n\nshell reported `cap $200`\n\n— the figure from before the change, not the current one. My\n\nshell had `MESH_LABOR_BUDGET_USD=200`\n\nexported from earlier in the session, and that\n\nexport shadowed the value `mesh-labor`\n\nreads from the config file. It's the exact\n\nfailure mode described two sections up, hit firsthand while checking whether that\n\nsection still held: the tool wasn't wrong, my terminal was holding a stale answer to a\n\nquestion I'd already asked once this session. Re-sourcing the file fixed it. I'm leaving\n\nthe numbers above as originally captured rather than editing them in place, because the\n\ngap between them and tonight's is itself the point.\n\nIt is not a spend forecast — the window is rolling and retrospective, so it\n\ntells you what already happened in the last five hours, priced, not what's\n\nabout to happen in the next five. It is not a hard, provider-verified bill —\n\n\"imputed\" means the number is only as trustworthy as the rate table behind\n\nit, and that table has already needed a correction once (a pricing gap for\n\none model class inflated pre-fix history). And it is not a permission system\n\n— it gates *when new autonomous work starts*, not what a mind already running\n\nis allowed to do with the turn it's mid-way through.\n\nThe generalizable idea isn't \"add a budget to your agents.\" It's that a\n\ncoordination log you're already writing — task/claim/done, in whatever shape\n\nyour system uses — is a transaction log whether you treat it as one or not.\n\nThe moment you replay it into a ledger with a real invariant (two independent\n\ncomputations of \"what's open\" that have to agree), you get three things for\n\nfree that are usually built as three separate systems: an audit trail, a\n\nleak detector for the promises that never got kept, and — if you price one\n\naxis — a control input a throttle can act on.\n\nThe throttle is only interesting because it's a *closed* loop. A dashboard\n\nthat shows spend is a story. A gate that reads the same number and holds\n\ndispatch is a control system, and the difference showed up in the same\n\ncommit message that turned the gate on — not a screenshot I staged for this\n\npost.", "url": "https://wpnews.pro/news/we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day", "canonical_source": "https://dev.to/ilya_mozerov_867dbdd91feb/we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day-c59", "published_at": "2026-07-28 18:22:13+00:00", "updated_at": "2026-07-28 19:04:33.672003+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "artificial-intelligence"], "entities": ["hledger", "Anthropic", "Claude"], "alternates": {"html": "https://wpnews.pro/news/we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day", "markdown": "https://wpnews.pro/news/we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day.md", "text": "https://wpnews.pro/news/we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day.txt", "jsonld": "https://wpnews.pro/news/we-gave-our-ai-agent-fleet-a-credit-limit-and-it-hit-it-the-same-day.jsonld"}}