{"slug": "your-agent-bills-while-it-waits-here-s-the-fix", "title": "Your Agent Bills While It Waits. Here's the Fix.", "summary": "Hamza Tahir of ZenML highlighted that most of an agent's life is spent waiting on tools, humans, or next steps, while billing continues. A peer-reviewed study, 'The Harness Effect,' found that re-sent context accounts for 62% of agent inference bills, and CockroachDB confirmed this pattern at scale. The unit economics problem shows agentic workflows consume 5-30x more tokens per task than chatbots due to orchestration overhead, retries, and idle context maintenance.", "body_md": "📖\n\n[Read the full version with charts and embedded sources on AgentConn →]\"Most of an agent's life is spent waiting — on a tool, a human, the next step — and the whole time you're holding a live process awake and billing for it.\" — Hamza Tahir, ZenML, AI Engineer 2026\n\nThat single sentence, dropped in an [AI Engineer conference talk this week](https://www.youtube.com/watch?v=bZISsg7H7DA), names the unit-economics problem that no agent demo video ever mentions. The problem that decides which agent companies survive contact with a CFO.\n\nEvery agent product demo shows the same clip: the agent plans, calls tools, produces output — a continuous stream of productive work. What the demo never shows is what happens between those steps. The tool call that takes 8 seconds to return. The human-approval gate where someone is in a meeting. The retry backoff after a rate limit. The polling loop waiting for a webhook. The overnight batch job that parks a process for 6 hours.\n\nThat is where your money goes. Not inference. Not model selection. The *waiting*.\n\nA [peer-reviewed study published this month](https://arxiv.org/html/2607.06906v1) — \"The Harness Effect\" — ran controlled experiments across enterprise agent deployments and found something that should alarm every engineering leader running agents in production:\n\n[Read the full paper on arXiv →](https://arxiv.org/html/2607.06906v1)\n\nCockroachDB's engineering team [confirmed this pattern at scale](https://www.cockroachlabs.com/blog/agentic-ai-costs-at-scale/): re-sent context alone — system prompts, tool definitions, and state history redundantly transmitted across model calls — accounts for **62% of total agent inference bills** according to Stanford Digital Economy Lab research. Goldman Sachs projects a 24-fold increase in token consumption by 2030, meaning these inefficiencies compound, not shrink.\n\n[Read the full post on CockroachDB →](https://www.cockroachlabs.com/blog/agentic-ai-costs-at-scale/)\n\nℹ️\n\nThe unit economics in one line:Agentic workflows consume 5-30x more tokens per task than standard chatbots. The multiplier is not from better reasoning — it is from orchestration overhead, retries, and idle context maintenance.\n\nWhere does the waiting actually happen? Four categories dominate.\n\nEvery external tool invocation is a network round-trip. A [deep technical analysis of agent wall-clock budgets](https://tianpan.co/blog/2026-06-03-the-agent-wall-clock-budget-that-raced-your-tools-own-timeout) identified **four unsynchronized clocks** operating in parallel during a single tool call:\n\n[Read the full analysis on TianPan.co →](https://tianpan.co/blog/2026-06-03-the-agent-wall-clock-budget-that-raced-your-tools-own-timeout)\n\nHidden latency accumulates before tool execution even starts: TTFT delays, streaming pipeline hops, MCP router overhead, tool worker queue backlog. A tool with an 8-second budget may have only 6.6 seconds from the agent's perspective after 1.4 seconds of transit overhead.\n\nThe failure modes are worse than the latency: orphaned successful results (tool finished after the agent gave up), duplicate tool calls from retry logic, and agents replanning despite correct responses that arrived a beat too late.\n\nThe most expensive wait of all — and the most unpredictable. When an agent hits an approval step:\n\nDuring all of this, a conventional agent harness holds a live process open — memory allocated, event loop spinning, connection pools warm. You are billing for sleep.\n\nWhen a tool returns a 429 or a transient error, the standard pattern is exponential backoff: wait 1s, then 2s, then 4s, then 8s. During each wait, the process is alive and metered. Reddit practitioners [report that agents retry-looping on bad tool outputs quietly burn 5-10x the expected token budget](https://dev.to/lura_cardena_7de06f82aacd/ai-agents-on-reddit-late-april-to-early-may-2026-ten-threads-about-cost-reliability-and-real-4f20) per task before a human notices.\n\nAgents waiting for async results — CI pipeline completion, deployment status, file processing — often implement polling. Every poll iteration is a model turn that consumes tokens, even when the answer is \"not ready yet.\" Multiply by dozens of concurrent agents and you have a fleet burning budget on the computational equivalent of checking if the oven has preheated.\n\nThe fix is not \"make agents faster.\" The fix is **stop billing for the waiting.**\n\nDurable execution — the pattern implemented by [Temporal](https://temporal.io/), [Inngest](https://www.inngest.com/blog/durable-execution-key-to-harnessing-ai-agents), [Rivet Actors](https://github.com/rivet-dev/rivet), and now [Cloudflare Workflows](https://workers.cloudflare.com/product/workflows) — treats waiting as a **continuation** rather than a loop:\n\n[Read the full post on Inngest →](https://www.inngest.com/blog/durable-execution-key-to-harnessing-ai-agents)\n\nThe key insight from Inngest's engineering team: the `waitForEvent`\n\ncall suspends the workflow entirely — no compute resources consumed while waiting. Their data shows human-in-the-loop suspend/resume patterns **drop idle-pending cost by 60-80%.**\n\nMicrosoft's Agent Framework, [announced at BUILD 2026](https://devblogs.microsoft.com/agent-framework/microsoft-agent-framework-at-build-2026-announce/), built this principle in from day one: agents **scale to zero**, paying nothing while idle, and scale back up on the next request — with files, disk state, and session identity persisting across the scale-to-zero boundary. Their CodeAct pattern goes further: instead of choosing a tool, waiting, choosing the next — the model writes a single Python program that calls tools via `call_tool()`\n\n, runs it once in a sandbox, and returns a consolidated result. One round-trip instead of ten.\n\nThe durable execution category for agents is not theoretical — it is shipping. Here is the current landscape.\n\nTemporal's February 2026 Series D ($300M at $5B valuation, led by a16z) and its [Replay 2026 release wave](https://byteiota.com/temporal-replay-2026-serverless-workers-ai-agents/) — Workflow Streams for LLM output, Serverless Workers, an OpenAI Agents SDK integration GA since March — signal a company repositioning hard around agents.\n\n**Strengths:** Seven language SDKs, deepest production track record, event-history replay semantics.\n\n**Trade-off:** Operational complexity. You run the cluster (or pay for Temporal Cloud). Workflow versioning is genuinely hard.\n\nEvent-driven step functions with no stateful backend to operate. Their [AgentKit](https://www.inngest.com/blog/durable-execution-key-to-harnessing-ai-agents) is a first-party multi-agent framework with MCP tooling, backed by a $21M Series A (Altimeter, September 2025).\n\n**Strengths:** Zero infrastructure to manage. Step-function model maps cleanly to agent workflows. Built-in event triggers.\n\n**Trade-off:** Less flexibility than Temporal for complex orchestration. Newer, less battle-tested at scale.\n\n[Rivet Actors](https://github.com/rivet-dev/rivet) are the lowest-level option: stateful workload primitives built explicitly for AI agents, collaborative apps, and durable execution. Open-source under Apache 2.0 — you own the infrastructure completely.\n\nTheir companion project [AgentOS](https://github.com/rivet-dev/agentos) runs coding agents inside isolated Linux VMs with an in-process operating system kernel. Everything runs inside the kernel; nothing executes on the host. The pitch: faster, lighter, cheaper than mainstream sandbox providers.\n\n**Strengths:** Maximum control. No vendor lock-in. Purpose-built for the agent use case.\n\n**Trade-off:** You build more yourself. Less ecosystem than Temporal.\n\nYou cannot optimize what you cannot see. [SigNoz pivoted to agent-native observability](https://signoz.io/agent-native-observability/) — waterfall views of every model call, tool invocation, and reasoning step. Their [Claude Agent SDK integration](https://signoz.io/docs/claude-agent-monitoring/) via OpenTelemetry gives you P99 latency on tool calls, token budgets per session, and alerts on loops that exceed cost thresholds.\n\nThis is the missing telemetry: which tool calls are slow, which approval gates are the bottleneck, where retry storms happen, and how much money each idle period actually costs.\n\nLet us make this concrete. A production coding agent that:\n\n**Without durable execution:**\n\nScale that to 20 agents and you have **18.7 hours of paid idle time per day.**\n\n**With durable execution:**\n\nAt typical cloud pricing, that is the difference between \"agents are too expensive for anything but demos\" and \"agents pay for themselves.\"\n\nThe Hacker News thread [\"Improving 15 LLMs at Coding in One Afternoon — Only the Harness Changed\"](https://news.ycombinator.com/item?id=46988596) crystallized the consensus: the model and its harness form \"a whole cybernetic system of feedback loops\" where \"the harness can make as much if not more of a difference, when improved, as improvements to the model itself.\"\n\n[View the full discussion on Hacker News →](https://news.ycombinator.com/item?id=46988596)\n\nMeanwhile, another HN discussion — [\"Effective harnesses for long-running agents\"](https://news.ycombinator.com/item?id=46081704) — dug into what long-running agents actually need: multi-agent judge setups, external memory, context management, and crucially, state persistence across crashes.\n\n[View the full discussion on Hacker News →](https://news.ycombinator.com/item?id=46081704)\n\nThe thread [\"The agent harness belongs outside the sandbox\"](https://news.ycombinator.com/item?id=47990675) pushed even further: the harness itself — including its durable execution layer — must live outside any sandboxed environment, because the orchestration state needs to survive the sandbox tearing down.\n\n[View the full discussion on Hacker News →](https://news.ycombinator.com/item?id=47990675)\n\nOn X, [@naval](https://dev.to/naval) summarized the convergence: \"an agent harness that plays games, writes code, reasons like a physicist, and saturates the ARC-AGI-3 benchmark\" — underscoring that the harness IS the product surface now, not the model inside it.\n\nPractitioners are converging on the same conclusion from the cost side. The [AI Engineer conference this week](https://www.ai.engineer/worldsfair/2026) dumped four talks in a single day — feature flags, durable execution, signed receipts, and reward hacking — all arguing that agents lack the operational primitives web software got a decade ago.\n\n⚠️\n\nThe contrarian view:Durable execution adds real operational complexity — workflow versioning, replay semantics, state serialization, schema evolution. For teams running fewer than ~50 concurrent agents, the engineering overhead of Temporal may cost more in developer-hours than the idle compute it saves. The break-even is not zero. Know your scale before you adopt.\n\n**If you are building agents today:**\n\n**Instrument first.** Before you optimize, you need to see where time goes. Add OpenTelemetry traces to every tool call. Track wall-clock time per step. [SigNoz](https://signoz.io/agent-native-observability/) or any OTel-compatible backend will show you the idle-time breakdown.\n\n**Separate \"fast waits\" from \"slow waits.\"** Tool calls under 5 seconds are not worth checkpoint/restore overhead. Human approvals and retry storms over 10 seconds are where durable execution pays off immediately.\n\n**Choose your complexity budget.** Temporal if you are enterprise-scale and have platform engineers. Inngest if you want minimal ops burden. Rivet if you want to own the stack. Cloudflare Workflows if you are already in their ecosystem.\n\n**Budget per agent, not per model.** The [Harness Effect paper](https://arxiv.org/html/2607.06906v1) showed blended-model cost per task falling 41% from harness optimization alone. Track cost at the task level, not the API-call level. That is where the real unit economics live.\n\n**Set circuit breakers.** [Reddit practitioners are unanimous](https://dev.to/lura_cardena_7de06f82aacd/ai-agents-on-reddit-late-april-to-early-may-2026-ten-threads-about-cost-reliability-and-real-4f20): without MAX_LOOPS and per-task token ceilings, your agent will run until the billing alarm fires.\n\nℹ️\n\nThe prediction:Within 12 months, \"how does your agent handle idle time?\" will be a standard question in vendor evaluations and architecture reviews. The teams that treat it as an infrastructure problem — not a model-quality problem — will run 5-10x more agents at the same budget. That is the real competitive advantage of durable execution: not reliability (though you get that too), but unit economics that let you scale.\n\n*For more on harness engineering as the decisive layer, see our coverage of why the harness — not the model — determines agent reliability, memory as the new competitive moat, and observability for agent budgets.*\n\n*Originally published at AgentConn*", "url": "https://wpnews.pro/news/your-agent-bills-while-it-waits-here-s-the-fix", "canonical_source": "https://dev.to/max_quimby/your-agent-bills-while-it-waits-heres-the-fix-2g6m", "published_at": "2026-07-19 04:45:28+00:00", "updated_at": "2026-07-19 04:57:03.470197+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-infrastructure", "ai-products", "developer-tools"], "entities": ["Hamza Tahir", "ZenML", "CockroachDB", "Stanford Digital Economy Lab", "Goldman Sachs", "TianPan.co", "Reddit"], "alternates": {"html": "https://wpnews.pro/news/your-agent-bills-while-it-waits-here-s-the-fix", "markdown": "https://wpnews.pro/news/your-agent-bills-while-it-waits-here-s-the-fix.md", "text": "https://wpnews.pro/news/your-agent-bills-while-it-waits-here-s-the-fix.txt", "jsonld": "https://wpnews.pro/news/your-agent-bills-while-it-waits-here-s-the-fix.jsonld"}}