What a real product refactor revealed about AI coding agents

Aashish Bhandari led a refactor of the alpha Markdown-review application ReviewWithAI using two AI coding agents, with Goku acting as principal architect and implementing developer and Naruto independently reviewing the design and code. Telemetry analysis of the implementation conversation found that activations issuing waits for delegated workers accounted for more than a quarter of the parent agent's recorded tokens, and that worker reuse concentrated consumption across stages, though the report cautions the evidence does not establish whether fresh workers would have matched quality with less effort. The report concludes that coding agents' surrounding mechanisms must be evaluated alongside their output, and calls for better deterministic counters, evaluation budgets and quality-inclusive comparisons.

ReviewWithAI engineering case study · 15–16 September 2026 By Aashish Bhandari, with AI-assisted analysis by Goku Codex, Astra and independent review by Naruto Claude, Fable 5.1 . I worked with two AI coding agents to refactor ReviewWithAI, an alpha application for reviewing Markdown documents and handing changes to external agents. Goku acted as principal architect and implementing developer. Naruto independently reviewed the design and code. I set priorities, resolved material decisions and authorized progress through the review checkpoints. The work implemented eleven low-level designs across three milestones. It covered server and browser structure, authorization, persistence, testing, operational diagnostics and release preparation. The candidate passed independent checks, including tests, browser workflows and reproduction of the packaged application. Those results establish a reviewed engineering outcome; they do not establish production readiness or prove that the process was efficient. After implementation, we examined the runtime evidence to understand how the work had been orchestrated. The analysis used collected telemetry first, followed by a bounded deterministic inspection of fields missing from the collector. Its scope was Goku's implementation conversation, delegated workers and automatic approval reviewers. Naruto's separate review sessions and my time were outside that measurement. The clearest finding concerned waiting. When the parent agent had no independent work, it repeatedly resumed after short waits for its workers. Each resumption could carry conversation history back into a model invocation. Activations that issued waits accounted for more than a quarter of the parent's recorded tokens. Some waits returned completed work, so that share cannot simply be called waste or promised as recoverable savings. A second finding concerned worker reuse. Several delegated conversations accumulated assignments across multiple stages. Their recorded consumption became concentrated, but the evidence does not tell us whether starting fresh workers would have produced the same quality with less effort. Continuity may help correctness while increasing context costs. That tradeoff needs a controlled comparison. Compaction reduced the context entering the next invocation, but later context growth and recurring waits continued. Choosing cheaper worker models also lowered a calculation based on published rates, without demonstrating lower total work. Most recorded input was cached. Processed tokens therefore need to be distinguished from unique content, uncached computation and actual charges. The investigation exposed limitations in our own measurement process. The collector omitted compaction activity, and its interruption counter did not describe every terminal failure. Automatic approval reviewers also consumed resources separately from implementation workers. Even evaluating the session had a material cost: the analysis conversation included other work, preventing a clean estimate of analysis overhead. We report that uncertainty rather than presenting precision. The practical lesson is to evaluate the mechanisms around coding agents alongside their output. Instructions to avoid polling did not reliably prevent the observed behavior. Before testing optimizations, we need better deterministic counters, evaluation budgets and comparisons that include accepted quality, recovery and human effort. This report is a baseline from one project, with delivery and unresolved efficiency questions. The next step is to test changes against that baseline and publish the results, including any changes that fail to help. The detailed evidence below lets other developers inspect our reasoning and judge where these observations might apply to their workflows. Boring stuff ahead. Proceed with caution. The rest contains the methods, tables and caveats for readers who want to check the evidence. ReviewWithAI is a Markdown review application: a user selects text, attaches comments, hands work to an external coding agent, checks changed anchors, records repairs and accepts a specific source revision. The starting point was an alpha-grade, mid-sized product, as described by its developer. The work included server behavior, browser interactions, persistence, authorization, testing, documentation and release tooling. It was an existing-product refactor and remediation effort, rather than initial application generation. The eleven low-level designs LLDs addressed twelve review findings labelled Q1–Q12; Q3 and Q4 shared one design. Design preparation and agreement preceded the measured Phase B implementation task. The roles were: | Participant | Responsibility | Included in the measured task? | |---|---|---| | One human developer, Aashish Max | Product direction, scope, material decisions, milestone authorization and final candidate acceptance | Human time is not measured | | Goku, Codex parent on GPT-6 Astra | Principal architect, implementing developer, delegation, integration and verification | Yes | | Sixteen Goku worker threads on Sol, Luna and Terra | Bounded research, implementation, integration, tests and release work; some threads were later reused | Yes | | Naruto, Claude coding agent; final review identifies Fable 5.1 | Independent principal architect for design and code review, including reproduction of delivery evidence | No; separate review sessions | | Automatic Codex approval reviewers | Review of eligible approval requests | Yes; separate from implementation workers | “Two agents” describes the two principal collaborators. It does not mean two model processes: Goku's implementation task alone produced sixteen worker threads and 38 approval-review components. php flowchart TD H "Human developer: scope, decisions and acceptance" -- G "Goku: architecture and implementation" H -- N "Naruto: independent design and code review" subgraph measured "Measured Codex task" G -- W "16 delegated worker threads" G -. "approval requests across the task" .- A "38 approval-review components" end G -- E "Candidate code, package and verification evidence" W -- E E -- N N -- R "Review findings and milestone verdicts" R -- H R -- G The milestones were review checkpoints rather than individual commits. The LLD index and checkpoint records //../../SOURCE REFERENCES.md ref-10 establish the sequence: | Milestone | Implemented scope | Review boundary | |---|---|---| | 1. Housekeeping and engineering controls | Engineering-standard amendments, LLD navigation and citations, documentation checker and checker tests | Human and Naruto approval before merge and runtime implementation | | 2. First implementation group | Q5 handoff-route tests; Q10 transaction guard; combined Q3/Q4 typed operations, browser structure, retries and credential rotation | Independent midpoint review; approved work integrated at c9215c9 | | 3. Remaining implementation and release candidate | Q2 diagnostics; Q6 portable tooling; Q8 strict agent inputs; Q9 health version; Q11 bounded discovery; Q12 handoff provenance; Q7 contributor docs; Q1 release curation | Final independent closure review after integration, packaging and artifact checks | Formatting and documentation-baseline amendments arrived after the initial Q1 freeze. They were applied and release curation was repeated at 64aa194 ; the final review explicitly accepted that sequence deviation. This additional work is included in the task's recorded consumption. | Design | Finding s | Engineering change | |---|---|---| | Handoff endpoint verification | Q5 | Seven route tests and expiry sensitivity | | Non-nested transaction contract | Q10 | Explicit transaction ownership, rollback and cleanup behavior | | Structural refactor | Q3, Q4 | Typed handlers, browser decomposition, shared retry and credential-rotation ownership | | Operational diagnostics | Q2 | Redacted, correlated and rate-bounded diagnostics | | Portable tooling | Q6 | Browser checks and separate source/script typechecks | | Strict agent inputs | Q8 | Shared action policy and strict integer validation | | Health version | Q9 | Health response derived from package version | | Bounded document discovery | Q11 | Short-lived cache with capacity and recovery boundaries | | Handoff provenance | Q12 | Persisted provenance, authorization checks and schema migration | | Public contributor documentation | Q7 | Accurate contributor guides and explicit documentation gates | | Release curation | Q1 | Curated source history, portable evidence and reproducible package | The target was the closed task Complete Phase B Design fixes for Q1 to Q12 , parent 01a0a59d-ca16-76b3-8a9a-5f95389d22e3 . The analysis included its sixteen worker threads and associated approval reviewers. Prior design work, Naruto's independent review sessions, human effort and subsequent evaluation/editorial work fall outside that population. Routine totals come from the canonical derived record //../../../telemetry/ai-review-v2/codex/main/sessions/01a0a59d-ca16-76b3-8a9a-5f95389d22e3.json , captured by codex-hook-v1 and closed by SessionEnd at 2026-09-16 04:13:19 UTC. The derived record does not retain worker task names, wait outcomes, compaction boundaries or terminal error reasons. A targeted accuracy audit therefore used deterministic jq , rg , find , Git and arithmetic against only this parent's 15,355,576-byte approximately 15.4 MB runtime file and selected child metadata. No model worker parsed logs. Prompt bodies, source bodies, secrets and raw output were not copied into this report. Product outcome and authorship were checked against the contribution record //../../SOURCE REFERENCES.md ref-5 , Goku's final response //../../SOURCE REFERENCES.md ref-9 and Naruto's final review //../../SOURCE REFERENCES.md ref-8 . Product files remained read-only. The measured wall window is 47,022 seconds, from 15 September 15:09:37 UTC to 16 September 04:13:19 UTC. It includes a roughly 7h33m gap between completed parent turns, external test waits and user pauses; it is not active model time. | Term | Meaning in this report | |---|---| | Activation | A recorded model invocation, rather than a user prompt or a tool call | | Processed tokens | Cached input + uncached input + output; cumulative across invocations | | Cached input | Input tokens served from the model's prompt cache; still included in processed-token accounting | | Output | Recorded output tokens, including reasoning where the counter exposes it | | Worker thread | A delegated conversation that may contain multiple follow-up assignments | | Guardian | An automatic approval-review component, distinct from a worker-health watchdog | | Wait-generating activation | A parent invocation that issued a wait; its entire usage is associated with that invocation, not isolated instruction-by-instruction | | Frozen price equivalent | Recorded token categories multiplied by a dated API rate schedule; an analytical comparison rather than a measured subscription charge | Repeatedly presenting a large context contributes to the cumulative token count even when much of that input is cached. A 158M-token total therefore does not imply 158M tokens of unique code, prose or uncached processing. The analysis proceeded from derived aggregates to component/model reconciliation, then to a targeted deterministic audit of missing orchestration fields, and finally to comparison with independent product-review evidence. This editorial revision uses those existing findings; it does not reopen the raw transcripts or rerun the product tests. The compared Git range 69686a3..bb7af7d contains 27 commits and changes 118 files: 16,369 insertions and 3,229 deletions. The engineering-file subset covers 46 files, 11,903 insertions and 2,767 deletions. These figures include tests and tooling and the full range also includes reviews and records; neither is a pure backend-line count. Naruto's independent closure review found no P0 or P1 and reproduced: git diff --check ; 485f87aceaa5f4bebf7c2c49e7d55d44b68d1cab00fec1de5fc29442db911ae3 . These are candidate-level checks. They do not establish production reliability or absence of defects. Naruto recorded five non-blocking observations, including handoff-citation semantics, package reproduction instructions and retained module-size debt. Human candidate acceptance and publication remained separate decisions at the final-review snapshot. The collector records 1,299 tool calls across the measured task. Token and activation totals are separated below by component; the total excludes Naruto's separate review sessions and the later performance analysis. | Component class | Components | Activations | Processed | Cached input | Uncached input | Output | Share | |---|---|---|---|---|---|---|---| | Parent Astra | 1 | 342 | 36,090,696 | 34,588,032 | 1,337,219 | 165,445 | 22.82% | | Worker threads | 16 | 1,011 | 110,255,046 | 106,565,888 | 3,211,069 | 478,089 | 69.72% | | Approval-review guardians | 38, of which 20 non-zero | 152 | 11,791,577 | 9,690,112 | 2,081,891 | 19,574 | 7.46% | | Total | 55 | 1,505 | 158,137,319 | 150,844,032 | 6,630,179 | 663,108 | 100% | Reasoning output is a 238,390-token subset of output, not an additional category. Cache writes are zero. Processed tokens are not bytes transmitted, inference FLOPs, energy, quota usage or a subscription bill. Rates are the dossier's 15 September 2026 frozen standard API rates. They may change. codex-auto-review has no recorded public rate and is excluded from dollars. | Model | Components | Activations | Processed | Share | Frozen standard equivalent | |---|---|---|---|---|---| | GPT-6 Astra | 1 | 342 | 36,090,696 | 22.82% | $56.23 | | GPT-5.6 Sol | 9 | 592 | 69,756,461 | 44.11% | $42.21 | | GPT-5.6 Luna | 6 | 414 | 40,236,736 | 25.44% | $1.15 | | GPT-5.6 Terra | 1 | 5 | 261,849 | 0.17% | $0.21 | | codex-auto-review | 20 non-zero | 152 | 11,791,577 | 7.46% | Unknown | Named-model total: 146,345,742 tokens and $99.80 . Pricing the same named tokens entirely as Astra gives $218.81 , a 54.39% price substitution reduction . Worker-only usage is $43.57 at the actual model mix versus $162.58 at Astra rates, a 73.20% price substitution reduction . The frozen calculation assigns a lower price to the recorded model mix. It holds token counts constant and supplies no evidence about the amount of work an actual all-Astra run would perform; Sol and Luna together processed 109.99M tokens in the observed run. Static task names became incomplete descriptions because the parent issued 28 followup task calls and 24 send message calls . The “actual work” column follows the completed worker-turn records and product contribution evidence, not the original label alone. | Worker label | Model | Actual observed work and terminal state | Processed | Activations | Cache read | |---|---|---|---|---|---| | design evidence | Terra | Read-only Q1-Q12 evidence map; no files changed | 261,849 | 5 | 73.81% | | commit1 standard | Sol | B1-B4 engineering-standard document | 246,654 | 6 | 73.59% | | commit1 checker | Sol | Documentation checker, tests and bounded corrections | 1,500,331 | 24 | 94.83% | | commit1 navigation | Luna | Eleven-LLD index, plan link and citation corrections | 559,111 | 12 | 89.74% | | commit1 gate | Luna | Commit-1 integrated gate; 56/56 tests | 525,185 | 15 | 95.20% | | midpoint merge | Sol | Reused for Q10 transaction guard and regressions | 1,871,363 | 34 | 96.66% | | q5 handoff | Sol | Seven Q5 handoff-route tests and expiry sensitivity | 2,775,134 | 34 | 96.53% | | q34 browser | Sol | Q3/Q4 browser split, mutation lifecycle and regressions | 9,609,041 | 81 | 98.07% | | q34 server | Sol | Q3/Q4 server types, handlers, retry and credential rotation | 9,473,162 | 68 | 96.72% | | midpoint gates | Luna | Reused for integration, verification and midpoint records | 3,968,535 | 68 | 93.92% | | final merge | Luna | Rebase/integration, Q2-Q12 gates, packaging, two long artifact runs and final freeze | 33,364,102 | 287 | 97.85% | | q6 portability | Sol | Reused for Q6, Q12 and B-F/B-D baseline work; final turn ended at model capacity after earlier deliverables | 20,249,883 | 165 | 96.05% | | q2 diagnostics | Sol | Reused for Q2, Q8, Q9 and integrated Q11 work | 10,394,239 | 83 | 96.43% | | q11 discovery | Sol | Reused for Q11, Q1 tooling and curation; final closing-record turn ended at model capacity after earlier deliverables | 13,636,654 | 97 | 96.51% | | closing records | Luna | Six-document closing packet | 918,820 | 15 | 91.19% | | closure merge | Luna | Final fast-forward while preserving five local edits | 900,983 | 17 | 94.19% | Four threads— final merge , q6 portability , q11 discovery and q2 diagnostics —were used across multiple stages and together consumed 77,644,878 tokens: 70.42% of worker usage and 49.10% of the task. A follow-up can carry accumulated conversation context into later invocations. The aggregate concentration is observed; the fraction caused by reuse, and the benefit of starting fresh workers, remain unmeasured. Two Sol threads ended their last follow-up with Selected model is at capacity after completing earlier stages. The derived record still reports zero interruptions. This is not a contradiction in the current schema—capacity failure is not counted as interruptions —but it is a telemetry gap: component terminal outcomes should be explicit rather than inferred from raw child events. The approved telemetry-extension work, ED-6, is the relevant workstream. Its current enumerated fields do not explicitly include terminal outcomes or follow-up counts; those additions need an explicit specification rather than being described as already installed or already enumerated in the decision. The parent recorded 330 tool calls: | Parent action | Calls | |---|---| | Shell/tool execution | 170 | | Spawn worker | 16 | | Follow-up assignment | 28 | | Worker message/steering | 24 | | wait agent | 89 | | sleep | 3 | Orchestration and waiting were 160 of 330 tool calls, 48.48% . Tool-call count is not token cost, but it demonstrates how much of the parent's control flow became coordination rather than direct engineering. The canonical lifecycle record contains 12 parent Stop events. Dividing 342 parent activations by those events gives 28.5 activations per recorded stop . This is a descriptive average, not a per-prompt compliance result: it hides the distribution and uses lifecycle events as its denominator. wait agent : 89 calls; 88 × 60 seconds, 1 × 10 seconds. The largest parent work turn issued 62 waits, 50 of which timed out. Those wait-generating activations consumed 7,443,509 tokens. Another turn issued 23 waits, 14 timeouts, consuming 2,377,824 tokens. The runtime also recorded 42 SubagentStop lifecycle events while only 23 waits returned completion-like output. These counts are not one-to-one because one worker thread can run multiple follow-ups and events can batch, but they establish that completion delivery existed independently of a fresh one-minute status check. The polling loop was not the only way completions reached the parent. Finding: the global declarative no-poll guidance did not enforce event-driven behavior when the parent had no independent work. This task reproduces the previously observed active-versus-idle split at much larger scale. The causal contribution of platform instructions versus parent choice is not isolated here. php flowchart LR A "Parent activation" -- B "Issue wait: usually 60 seconds" B -- C{"Wait result"} C -- |"Timeout: 66 returns"| D "Parent resumes; may issue another wait" D -- A C -- |"Completion-like: 23 returns"| E "Process worker result" The diagram shows the observed return categories; it does not count uninterrupted timeout-to-wait sequences. 66 timeouts is not automatically 66 repeated waits with no intervening work. The minimum requested timeout was 10 seconds , while the dominant cadence was 60 seconds. The 10.28M tokens and $12.72 cover all 89 wait-generating activations, including waits that returned completion-like output. Multiplying 66 by the all-wait mean of 115,528 gives approximately 7.6M tokens, but that estimate is neither an exact timeout-only total nor an upper bound on causal savings. An event-driven replacement still has to process completion, failures and recovery. Five runtime compactions occurred. Turn boundaries indicate that numbers 1 and 3 happened inside long work turns and were likely automatic; numbers 2, 4 and 5 were separate compaction turns and were likely manual. That classification is an inference; the raw compaction record does not label its trigger. | Compaction | Input before | First input after | Immediate drop | |---|---|---|---| | 1 | 191,950 | 32,181 | 83.23% | | 2 | 138,145 | 33,523 | 75.73% | | 3 | 209,562 | 34,271 | 83.65% | | 4 | 128,216 | 32,921 | 74.32% | | 5 | 50,518 | 32,097 | 36.46% | Compaction clearly reduced the immediate context carried into the next activation. It did not reduce the need for architectural boundaries: the parent later rebuilt a large context, and the post-compaction long turn alone consumed 20.43M raw parent tokens while issuing 62 waits. The parent raw ledger contains 347 token-usage records totalling 36,832,258 tokens; the canonical parent component contains 342 activations and 36,090,696 tokens. The 741,562-token difference aligns with the five compaction activations omitted by the derived parent aggregate. Therefore capture completeness: complete means all expected worker components were found; it does not mean all model work is represented. Whole-task model work is at least 158,878,881 tokens , before checking compaction omissions inside workers and guardians. That broader raw reconstruction was deliberately stopped rather than assigning a model to parse every child log. The collector associated 38 guardian components with this logical chat: codex-auto-review components: 11,791,577 processed tokens and 152 activations; These are low-effort automatic approval reviewers. They have zero tool calls and one turn per approval decision. They are not watchdogs supervising worker health, and their presence must not be used to claim watchdog coverage. Their price is unknown, but their 7.46% token share is operationally material. The review identified approval configuration as a further investigation: determine which approval requests caused these invocations and how the reviewer setting affected them. This report does not establish the task's configuration-to-cost relationship or evaluate the safety and human-review cost of changing that configuration. The earlier isolated implementation audit counted 24,995,374 full-scope tokens after adding compaction and approval-review usage. This Phase B task's 158,137,319 canonical tokens are 6.33× that historical build total. The workloads are not matched: Phase B covered 12 findings, repeated integration, packaging, long artifact tests, curation and closing records. The earlier efficiency-analysis task captured 14,932,980 tokens. The owner remembers it as roughly half the original task; comparing that snapshot with the 24,995,374 full-scope build gives 59.74%, but the historical review warned that the populations were not strictly aligned. Within that analysis, the Sol worker consumed 7,535,274 tokens, 50.46% of the analysis total. The owner's clarification supplies the reported workflow cause: Sol was asked to do log parsing after the existing Python helpers failed to produce the needed answer. That causal account is owner-supplied; this report did not repeat the old raw-log investigation. The approved recovery decision, ED-2 //../../../DECISIONS.md ed-2-instrumentation-gaps-and-audit-budget , requires a budget check before starting, a stop and costed script-change proposal when instrumentation is insufficient, and no ad hoc raw reconstruction inside the requested analysis. If estimated work exceeds the budget, approval comes before starting. If a running task crosses the budget, it ends with completed work and a costed proposal for the remainder; the human decides in the next message. The original Phase B analysis used derived telemetry plus bounded deterministic extraction from one parent file and selected child metadata, with no analysis subagent. That describes its method; it does not establish compliance with the approved workflow or budget. The earlier sentence claiming that this report “followed that rule” has been removed. Naruto's review identified a missing measurement: the original report did not disclose its enclosing analysis session's usage. That session also performed quick-reference preservation work, so derived aggregates cannot isolate the cost of writing the report. | Snapshot of Goku's evaluation session | Processed | Cached input | Uncached input | Output | Share of assessed task | |---|---|---|---|---|---| | Review snapshot, 16 September 08:07:40 UTC | 7,420,764 | 6,378,112 | 982,366 | 60,286 | 4.69% | | Later snapshot, 16 September 08:21:09 UTC, after the response to Naruto's review | 8,746,798 | 7,428,864 | 1,253,044 | 64,890 | 5.53% | The first snapshot records Sol only, 62 activations, 55 tool calls, seven Stop events and no workers. Reasoning accounts for 26,541 of its output tokens. The later snapshot records 74 activations, 66 tool calls, eight Stop events and no workers; reasoning is 29,018 of its output tokens. The later record is available in the canonical evaluation session //../../../telemetry/ai-review-v2-performance/codex/main/sessions/01a0a906-7e54-7770-bb20-9730aeb2e658.json ; the earlier counters were read and confirmed during the review response. This active record evolves, so it should not be expected to retain either snapshot as its latest total. Both ratios use the same 158,137,319-token assessed-task denominator. They are gross enclosing-session ratios, not isolated analysis overhead. The first is a conservative envelope for analysis work represented by that snapshot, subject to collector completeness. Neither includes the editorial work producing this revision or Naruto's cost of reviewing the analysis. The approved audit target, WC-7, is min 2% of assessed tokens, 1,000,000 — 1,000,000 tokens for this task . The first enclosing-session snapshot is 7.42 times that target. Enforcement was not installed at the review snapshot; the mixed session prevents an exact analysis-only compliance calculation. Having no analysis worker is an observed orchestration fact, not proof of low evaluation cost. The historical approximately 60% ratio and this 4.69% snapshot have different populations and coverage. They are useful context, but do not establish a controlled reduction in evaluation overhead. The existing decision register //../../../DECISIONS.md separates approved direction from installed mechanisms. At this case-study snapshot, ED-1 approved an interim five-minute Codex wait; ED-4/ED-5 approved a deterministic watchdog; ED-2 approved the audit budget and instrumentation-gap stopping rule; and ED-6 approved telemetry extensions. This report implements none of them. A deterministic watchdog can inspect counters without invoking an LLM, but still consumes CPU, memory and I/O. | Follow-up | Reason for examining it | Evidence needed | |---|---|---| | Extend the collectors before another comparison | Routine fields should not require another raw-log audit | Derived wait counts/outcomes/repeats, compaction usage, context bands and dated price calculations; explicitly specify additional lineage, follow-up, terminal-error and external-wait fields | | Address recurring parent waits first | All wait-generating activations account for 6.50% of recorded tokens and 12.74% of the known-price equivalent | Matched active/idle-parent scenarios and reliable completion, failure and lost-notification handling | | Bound worker follow-ups and activations | Four reused workers account for 49.10% of recorded tokens | A comparison of retained versus fresh worker context, including rework and accepted quality | | Examine approval-review configuration | Guardians account for 7.46% of recorded tokens at unknown price | Trigger attribution, useful review outcomes, latency and security consequences of any alternative | | Budget the next evaluation before execution | The evaluator's own session cost is material | A declared budget, bounded measurement scope, stop behavior and a separately reported evaluation total | The ordering reflects measurement confidence and implementability as well as price. Worker concentration and wait-generating activations measure different things; neither estimates removable waste. The large Luna final merge thread has a frozen price equivalent below one dollar, illustrating why token and price rankings can differ while neither captures latency or review effort. Validation would also need long external work, runaway behavior, parent restart and model-capacity failure cases. It should record total/cached/uncached/output tokens, elapsed time, guardian usage, rework and accepted quality. Fresh-worker benefits and net savings from event-driven orchestration remain hypotheses until that comparison exists. These rows describe Phase B under the existing winning-criteria definitions //../../../WINNING-CRITERIA.md . They do not update or approve the shared criteria. | Criterion | Defensible baseline from this analysis | Limit | |---|---|---| | WC-1: repeated waits without other work | 89 waits and 66 timeouts; exact qualifying repeat count uncomputed | Timeout count cannot substitute for sequence classification | | WC-2: smallest requested agent-wait timeout | 10 seconds; 88 of 89 requests used 60 seconds | Both are below the approved interim 300-second target | | WC-5: parent step budget | 342 activations / 12 recorded stops = 28.5 average | Does not establish the fraction of prompts within twelve steps | | WC-7: audit budget | 7.42M / 4.69% at the first review snapshot; 1M target | Enclosing session includes other work; later review/editorial cost must be reported separately | | WC-9: complete counting | At least 741,562 parent compaction tokens omitted | Worker and guardian compaction omissions not fully audited | LOCAL-CONFIRMED for the named closed session. This is a single-task observational case study. There is no matched alternative orchestration run, and no claim of measured equal-quality savings. Product tests were not rerun for this performance report. No collector, raw log, product file, hook, global instruction or runtime policy was modified. Original runtime analysis: Goku, recorded as GPT-5.6 Sol. The engineering task being measured used Goku on GPT-6 Astra. Model names follow recorded identifiers. This revision incorporates Naruto's analysis review and the human developer's clarification of project context. The tables above retain the original component totals, all sixteen worker rows, the five compaction boundaries, the Git-range measurements and the independent delivery checks. Source links are local evidence references within the evaluation dossier, not claims that the underlying private records are publicly accessible. | Evidence | What it supports | |---|---| | Closed implementation telemetry //../../../telemetry/ai-review-v2/codex/main/sessions/01a0a59d-ca16-76b3-8a9a-5f95389d22e3.json | Canonical component, token, activation and lifecycle totals | | Product contribution record //../../SOURCE REFERENCES.md ref-5 | Actual engineering contributions and model roles | | LLD and milestone source references //../../SOURCE REFERENCES.md ref-10 | Eleven designs, execution order and checkpoint boundaries | | Implementation response //../../SOURCE REFERENCES.md ref-9 | Finding dispositions, implementation gates and candidate evidence | | Naruto's final product review //../../SOURCE REFERENCES.md ref-8 | Independent reproduction, closure verdict and remaining observations | | Frozen pricing analysis //2026-09-15-goku-mechanism-followup.md | Dated rate basis and price-substitution method | | Decision register //../../../DECISIONS.md and criteria //../../../WINNING-CRITERIA.md | Approved direction and measurement definitions, separate from installation status | Post-review corrections: the original analysis byline changed from Astra to recorded Sol; evaluator-session consumption was added with dated cutoffs; ED-2 wording was aligned; wait timeouts, repeats and savings were separated; the minimum wait was corrected to 10 seconds for WC-2; and the uninstalled collector extensions were distinguished from proposed additional fields. The human developer supplied the one-developer/two-principal-agent framing and the alpha/mid-sized product description. Naruto's independently authored reviews remain unchanged.