{"slug": "i-profiled-the-agent-rebuild-ate-the-clock", "title": "I Profiled the Agent. Rebuild Ate the Clock.", "summary": "A developer built an open-source Python harness that profiles agent-loop latency by instrumenting four separate spans — serialization, tool execution, prompt rebuild, and model inference — and logging them to a CSV. Running the harness against a stubbed model revealed that quadratic prompt rebuilding, not model inference, dominated wall-clock time in later rounds, contradicting the common assumption that the language model is the bottleneck.", "body_md": "The language model was not my real bottleneck. Prompt rebuild ate the clock on every later round. I spent days blaming inference like a fool.\n\nHave you ever tuned the model and missed the copy? I wanted one graph I could actually keep. A zoo of dashboards would have hidden the crossing.\n\nI plotted rebuild time against a stubbed model nap. This is a lab note, not a customer war story. I did not harvest production traces for this.\n\nI wrote a harness you can run tonight. The plot is an agent loop with tools. Each tool returns a chunky JSON payload on purpose.\n\nThen the loop concatenates history into the next prompt. After that step it calls the model again. In the stub, the model is just sleep.\n\nOn the wire, the model is a network hop. The span names stay identical either way though. Why named spans and not a single timer?\n\nAverages already lied to me enough last month. I wanted stackable intervals with very boring names. I clocked serialize, tool, rebuild, and model separately.\n\nThey all land in one boring CSV file. The CSV is the graph I kept. Here is the core of that harness.\n\n``` bash\n#!/usr/bin/env python3\n\"\"\"Agent-loop span profiler. Stub model. Rebuild grows.\"\"\"\nfrom __future__ import annotations\n\nimport csv\nimport json\nimport os\nimport time\nfrom dataclasses import dataclass\n\n@dataclass\nclass RoundRow:\n    n: int\n    serialize_ms: float\n    tool_ms: float\n    rebuild_ms: float\n    model_ms: float\n    prompt_chars: int\n\ndef fake_tool(round_n: int) -> tuple[dict, float]:\n    t0 = time.perf_counter()\n    time.sleep(0.005)  # 5ms stand-in for a local tool\n    payload = {\n        \"round\": round_n,\n        \"files\": [\n            {\"path\": f\"src/mod_{i}.py\", \"preview\": \"x\" * 2000}\n            for i in range(50)\n        ],\n        \"log\": \"ok\" * 400,\n    }\n    return payload, (time.perf_counter() - t0) * 1000.0\n\ndef serialize_tool_result(payload: dict) -> tuple[str, float]:\n    t0 = time.perf_counter()\n    text = json.dumps(payload, separators=(\",\", \":\"))\n    return text, (time.perf_counter() - t0) * 1000.0\n\ndef rebuild_prompt(history: list[str], tool_text: str) -> tuple[str, float]:\n    t0 = time.perf_counter()\n    history.append(\"TOOL:\" + tool_text)\n    prompt = \"\"\n    # Quadratic on purpose: a join people actually write.\n    for i in range(len(history)):\n        prompt = \"\\n\".join(history[: i + 1])\n    return prompt, (time.perf_counter() - t0) * 1000.0\n\ndef stub_model(prompt: str) -> tuple[str, float]:\n    t0 = time.perf_counter()\n    time.sleep(0.040)  # ruler, not a model benchmark\n    _ = len(prompt)\n    return \"call_tool\", (time.perf_counter() - t0) * 1000.0\n\ndef run_loop(rounds: int, model_fn) -> list[RoundRow]:\n    history = [\"SYS:you are a coding agent\", \"USER:find the leak\"]\n    rows: list[RoundRow] = []\n    for n in range(1, rounds + 1):\n        payload, tool_ms = fake_tool(n)\n        tool_text, ser_ms = serialize_tool_result(payload)\n        prompt, rebuild_ms = rebuild_prompt(history, tool_text)\n        _, model_ms = model_fn(prompt)\n        rows.append(\n            RoundRow(n, ser_ms, tool_ms, rebuild_ms, model_ms, len(prompt))\n        )\n    return rows\n\ndef write_csv(path: str, rows: list[RoundRow]) -> None:\n    with open(path, \"w\", newline=\"\") as f:\n        w = csv.writer(f)\n        w.writerow(\n            [\"round\", \"serialize_ms\", \"tool_ms\", \"rebuild_ms\",\n             \"model_ms\", \"prompt_chars\"]\n        )\n        for r in rows:\n            w.writerow([\n                r.n,\n                f\"{r.serialize_ms:.3f}\",\n                f\"{r.tool_ms:.3f}\",\n                f\"{r.rebuild_ms:.3f}\",\n                f\"{r.model_ms:.3f}\",\n                r.prompt_chars,\n            ])\n\ndef ascii_graph(rows: list[RoundRow]) -> str:\n    lines = [\"round  rebuild#  model.\", \"-----  --------  ------\"]\n    for r in rows:\n        rb = max(1, int(round(r.rebuild_ms)))\n        md = max(1, int(round(r.model_ms / 2)))\n        lines.append(f\"{r.n:5d}  {'#' * rb}  {'.' * md}\")\n    return \"\\n\".join(lines)\n```\n\nI run the stub path with a plain command. Nothing fancy. No extra flags.\n\n```\npython agent_spans.py\n```\n\nWire `main` to `run_loop(12, stub_model)` and write `agent_spans.csv`. The stub sleeps for forty milliseconds each round. That sleep is a ruler, not a benchmark.\n\nPlease do not quote it as model speed. What should you watch on a quiet machine? Rebuild starts tiny, almost like a rounding error.\n\nThen the tool payload lands and history snowballs. By later rounds the copy work gets rude. The snowball is the whole point here.\n\nIf rebuild crosses that forty millisecond ruler, keep the graph. That crossing is the only picture I kept. Everything else was noise I threw away.\n\nI also wrap the same loop with cProfile. Spans tell a timeline, and functions tell a culprit. `json.dumps` often walks into the spotlight first here.\n\n``` php\nimport cProfile\nimport io\nimport pstats\n\ndef profile_loop() -> None:\n    pr = cProfile.Profile()\n    pr.enable()\n    run_loop(12, stub_model)\n    pr.disable()\n    buf = io.StringIO()\n    stats = pstats.Stats(pr, stream=buf)\n    stats.sort_stats(\"cumulative\").print_stats(15)\n    print(buf.getvalue())\n```\n\nDid you expect the model function to win? I did. It lost on the stub path.\n\nThat embarrassment is why I kept the picture. Export pstats if you want a flame-style view. A wide plateau sits under `rebuild_prompt` on stub runs.\n\nThe `stub_model` bar stays thin and honest. I still prefer the stacked CSV over a flame. It pastes into notes without a graphics stack.\n\nThe insult survives the copy-paste just fine. Sample stub output will look locally noisy. Treat the shape as the lesson, not the digits.\n\n```\nround  rebuild#  model.\n-----  --------  ------\n    1  ##        ....................\n    4  #####     ....................\n    8  ##########  ....................\n   12  ##################  ....................\n```\n\nYour bars will move. That is expected. Pin the payload size if you need a fair fight. Inflate the tool payload until the bars actually fight.\n\nNow comes the remote path with the same four spans. The model function POSTs the prompt over HTTP. I do not wrap a vendor SDK in this note.\n\nI pointed this harness at MonkeyCode because free model access and a free server option let the loop leave my laptop without a GPU box. Disclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nI am not publishing their latency, model names, or quotas. I am not calling this a product benchmark either. Point the client at any URL you actually control.\n\n``` php\nimport urllib.request\n\ndef remote_model(prompt: str) -> tuple[str, float]:\n    url = os.environ.get(\"AGENT_MODEL_URL\", \"\").strip()\n    if not url:\n        raise SystemExit(\"set AGENT_MODEL_URL to an endpoint you control\")\n    body = json.dumps({\"prompt\": prompt}).encode(\"utf-8\")\n    req = urllib.request.Request(url, data=body, method=\"POST\")\n    req.add_header(\"Content-Type\", \"application/json\")\n    t0 = time.perf_counter()\n    with urllib.request.urlopen(req, timeout=30) as resp:\n        raw = resp.read()\n    return raw.decode(\"utf-8\", errors=\"replace\")[:200], (time.perf_counter() - t0) * 1000.0\n```\n\nSet `AGENT_MODEL_URL` and run the same CSV path. Compare stub bars against wire bars on your graph. Keep the picture and throw away vendor adjectives.\n\n```\nexport AGENT_MODEL_URL=\"http://127.0.0.1:8080/prompt\"\npython -c \"from agent_spans import run_loop, remote_model, write_csv, ascii_graph\nrows = run_loop(8, remote_model)\nwrite_csv('agent_spans_wire.csv', rows)\nprint(ascii_graph(rows))\"\n```\n\nWatch the first remote round with extra suspicion. TLS and DNS can steal that opening lap. Queue delay on a shared box can steal more.\n\nAfter warmup, rebuild may still be the growth term. Why? You resend the whole history every round. Streaming would change this story quite a bit.\n\nThis harness does not stream a single token. If your real agent streams, instrument the decoder. Do not borrow my stub graph for that case.\n\nThe quadratic join is a microscope, not advice. Real history should be appended, not rebuilt from prefixes. I left the slow join in so the graph has teeth.\n\nSwap it for a single `\"\\n\".join(history)` after you see the crossing. Then run the CSV again on the same machine. Did the rebuild bar collapse like a cheap tent?\n\nIf it did, you just caught the copy. If it did not, stare at `json.dumps` next. Huge tool payloads serialize like wet cement.\n\nThis method will not find GPU kernel stalls. It will not find tokenizer weirdness on its own. cProfile also perturbs the thing you measure.\n\nLaptop clocks are noisy and fans make them worse. Run it thrice before you believe a crossing. Pin CPU frequency if you are being precious.\n\nA free shared server adds queue time you cannot see. Without server-side traces you must not blame the model. You can only blame the round trip you measured.\n\nSkip this if you already have tracing in the agent. Skip this if you need legal vendor comparisons. Skip this if your bottleneck is inside CUDA.\n\nI would also skip it for tiny prompts. A twenty token chat will not show rebuild. Inflate the tool payload until the bars fight.\n\nThat inflation is a lab trick, not production. Real tools should return less, not more. The harness is a microscope, not a product.\n\nClone the idea, not my sleep constant. Change the payload. Change the join. Keep one graph.\n\nWhen rebuild crosses your ruler, you have a bug. Fix the copy. Then go bother inference again.", "url": "https://wpnews.pro/news/i-profiled-the-agent-rebuild-ate-the-clock", "canonical_source": "https://dev.to/apppro_4800/i-profiled-the-agent-rebuild-ate-the-clock-4e0", "published_at": "2026-09-23 16:52:41+00:00", "updated_at": "2026-09-23 16:58:53.857100+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "large-language-models", "ai-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/i-profiled-the-agent-rebuild-ate-the-clock", "markdown": "https://wpnews.pro/news/i-profiled-the-agent-rebuild-ate-the-clock.md", "text": "https://wpnews.pro/news/i-profiled-the-agent-rebuild-ate-the-clock.txt", "jsonld": "https://wpnews.pro/news/i-profiled-the-agent-rebuild-ate-the-clock.jsonld"}}