{"slug": "decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the", "title": "Decoding AI’s Open-Source Course Maps Three Ways to Run an Agent Loop and the Provider Economics Behind Each", "summary": "LangChain's Terminal-Bench experiment found that changing only the harness—keeping the same model—moved a coding agent from roughly 30th place into the top 5, reframing harness design as an architecture decision. Paul Iusztin's open-source course 'Building a Coding Agent From Scratch' builds a Python agent called Decode with three run modes—interactive, remote, and async—each with different latency profiles and provider economics. The course, published through Decoding AI, separates the headless harness from interfaces and details how each mode suits a different inference provider based on latency and throughput needs.", "body_md": "Most teams treat ‘which model’ as the important decision. The harness engineering literature keeps pointing somewhere else. In [LangChain’s Terminal-Bench experiment](https://blog.langchain.com/the-anatomy-of-an-agent-harness/), changing only the harness—same model throughout—moved a coding agent from roughly 30th place into the top 5.\n\nThat result reframes the question. If the harness decides quality, then *how you run the loop* becomes an architecture decision, not a deployment detail. Paul Iusztin’s open-source course [Building a Coding Agent From Scratch](https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course) builds a Python agent called **Decode**. Published through [Decoding AI](https://www.decodingai.com/p/building-a-coding-agent-from-scratch-system-design), it separates three run modes. Each mode has a different latency profile. Each one therefore wants a different inference provider.\n\n**One headless core, three shapes**\n\nThe center of the system is a **headless harness** with no interface of its own. Inside it runs the agent loop every harness shares: the LLM picks an action, a tool executes, the observation feeds back. Everything reads from and writes to the context window.\n\nThe agent itself is small. In Decode it is a ~20-line [Pydantic AI](https://github.com/pydantic/pydantic-ai-harness) definition composing a model, tools, and an output type. In Claude Code’s leaked source, the core loop is roughly 150 lines. Everything else—memory, skills, sandbox, permissions, LSP feedback, compaction—is harness.\n\nInterfaces then plug into that core. **That is where the three modes appear:**\n\n**Mode 1: Interactive, online**\n\nA terminal UI is wired to one live session, in memory, in the same process. Events stream back through async generators as tokens arrive.\n\nThe hard problem here is steering. If you type while a tool call is in flight, injecting the message immediately corrupts the turn. Decode’s answer is a **steering queue plus a priority gate**. Input is buffered on arrival and injected only at a safe boundary. The loop exposes two: `MODEL_REQUEST`\n\n, before the next model call, and `WOULD_STOP`\n\n, when the turn would end.\n\nThree input modes map onto that. Plain `Enter`\n\nsteers within the turn. `Alt+Enter`\n\nqueues a follow-up until the turn stops. `Esc`\n\ntriggers a cooperative abort at the next boundary, clearing both queues so history stays intact.\n\nA human is reading every token. This mode is latency-bound, which is why it belongs on a low-latency hosted API.\n\n**Mode 2: Remote, offline**\n\nRemote mode keeps the harness headless and runs it on a server through an agent runtime. Decode uses [Kitaru](https://www.zenml.io/product/kitaru), ZenML’s agent runtime, deployed to GCP, with the agents themselves executing on [Modal](https://modal.com/).\n\nNobody is watching. A backlog of tickets fans out to N harnesses in parallel, each producing its own PR. Because the runtime records progress step by step, a sandbox that dies mid-task resumes from its last recorded step instead of restarting. A run that pauses for human input freezes and consumes no compute while it waits.\n\nTools execute inside [Modal Sandboxes](https://modal.com/docs/guide/sandboxes) remotely, Docker locally. The metric that matters is throughput per dollar, not time-to-first-token.\n\n**Mode 3: Async, online**\n\nThe third shape sits between the two. A live session hands work to a job queue and returns immediately. Background workflows fan out LLM calls and post results back later.\n\nThe user is online but not watching each step. The queue owns the work, so the run outlives the client that started it. This is the pattern behind Slack-triggered agents and background PR review, and it bills like batch, not like chat.\n\n**The interactive explainer**\n\n**Why the provider changes with the mode**\n\nThe cost model follows the latency requirement, and the gap is large.\n\nTake 1,000 documents at 30,000 input tokens each, roughly 500 output tokens per document. At frontier API rates of $3 per million input and $15 per million output, the lesson's arithmetic lands near **$97**. Prompt caching does not rescue it, because every document is a different prefix. Batched on a serverless GPU at around 3,000 tokens per second, the same work is under three hours of GPU time—roughly **$13**.\n\nThe reverse case is just as sharp. Decode's default test model, `Qwen3.6 35B`\n\n, runs on a single H200. [Modal's published pricing](https://modal.com/pricing) lists H200 SXM at $0.001261 per second, or about **$4.54 per hour**. Leave an interactive agent idle overnight waiting on a `y`\n\nconfirmation, and ten idle hours add roughly **$45** to the bill.\n\nThat is the whole argument. Interactive work pays per token because a human is waiting. Offline and async work pays per GPU-hour because throughput is the objective and idle time is the enemy.\n\nThere is a second axis: serverless versus reserved capacity. [Modal's pricing analysis](https://modal.com/blog/how-to-price-serverless) reduces it to one comparison. Reservations charge the peak rate for the whole contract; serverless follows the demand curve. When the **peak-to-average ratio exceeds the reservation discount**, serverless is cheaper. Modal reports typical discounts of 2–5× against peak-to-average ratios of 5–10× for inference, training, and agentic development. Industry surveys it cites put reservation utilization below 30%, often under 10%.\n\n**Key Takeaways**\n\n- Harness beats model: swapping only the harness moved an agent from ~30th to top 5 on Terminal-Bench.\n- Interactive mode is latency-bound and steers via a queue draining at\n`MODEL_REQUEST`\n\nand`WOULD_STOP`\n\nboundaries. - Remote and async modes are throughput-bound, so GPU-hour billing beats per-token billing at volume.\n- 1,000 documents cost ~$97 on frontier API rates versus ~$13 of batched GPU time.\n- Serverless wins whenever peak-to-average demand exceeds the reservation discount, typically 5–10× against 2–5×.\n\n**Sources:**\n\n[Building a Coding Agent From Scratch (Lesson 1)](https://www.decodingai.com/p/building-a-coding-agent-from-scratch-system-design)[The Bare-Bones Coding Agent Loop (Lesson 2)](https://www.decodingai.com/p/the-coding-agent-loop)[From a Raw Shell to a Sandboxed Coding Agent (Lesson 3)](https://www.decodingai.com/p/run-coding-agents-safely)[Course repository](https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course)·[Modal pricing](https://modal.com/pricing)[How to price serverless GPUs](https://modal.com/blog/how-to-price-serverless)[LangChain: The anatomy of an agent harness](https://blog.langchain.com/the-anatomy-of-an-agent-harness/)\n\nMichal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.", "url": "https://wpnews.pro/news/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the", "canonical_source": "https://www.marktechpost.com/2026/08/22/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the-provider-economics-behind-each/", "published_at": "2026-08-22 13:43:13+00:00", "updated_at": "2026-08-22 14:13:40.079275+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools", "ai-research"], "entities": ["LangChain", "Paul Iusztin", "Decoding AI", "Decode", "Pydantic AI", "Kitaru", "ZenML", "Modal"], "alternates": {"html": "https://wpnews.pro/news/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the", "markdown": "https://wpnews.pro/news/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the.md", "text": "https://wpnews.pro/news/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the.txt", "jsonld": "https://wpnews.pro/news/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the.jsonld"}}