{"slug": "bringup-notes-building-triton", "title": "Bringup Notes: Building Triton", "summary": "A developer building the Triton DirectX 11 driver for QEMU argues that AI tools amplify human developers rather than replace them, citing challenges such as sparse documentation, multi-boundary debugging, and difficulty quantifying success. The author emphasizes that the key skill for leveraging AI is the ability to break complex problems into smaller, AI-friendly tasks.", "body_md": "We [introduced Triton](/2026/introducing-triton-directx-11-driver-for-qemu/), a DirectX 11 driver for QEMU and you should read that other post about the technical and architectural details of the work. This post is focused on *how* we did the work. Just like with the [Neptune post](/2026/introducing-neptune-direct3d-virtualization-for-qemu/) I want to make an argument for how AI tools cannot replace developers but instead can amplify their work.\n\nTo set the stage, I need to explain *why* developing a DDI driver for virtualization is a *hard problem* (and why I had to lean on AI tools):\n\n- There is very little documentation and examples online of fully working WDDM drivers and DirectX DDI implementation. The definitive source of information comes from MSDN whose documentation on these topics are genuinely helpful locally (reading structure descriptions for example) but useless globally (understanding end-to-end flows). This means that it is much easier to debug a broken driver than it is to build one from scratch.\n- The work spans user/kernel boundaries, guest/host boundaries, and CPU/GPU boundaries. Each boundary on its own come with a set of debug challenges that gets exponentially more difficult when you combine them. Even the fact that the AI tool is running on a macOS host and constantly has to talk to a Windows guest was the source of many wasted turns.\n- Success can be hard to\n*quantify*. When Anthropic built their[C compiler](https://www.anthropic.com/engineering/building-c-compiler), they had a large corpus of inputs with a reference implementation to query in order to determine*correctness*. This kind of challenge can be brute forced by an LLM: try everything until the test passes. When building a graphics driver, you either need an existing collection of tests (such as the Vulkan CTS) or some A/B comparison of rendered frames. DirectX has some tests but they are no where near rich enough to build a driver just by observing test results. - Failure can be hard to\n*quantify*. How will the AI know that every other frame is being dropped? How will the AI know that some elements are drawn incorrectly? How will the AI know that while everything is rendered correctly and no frames are dropped, the performance is 60% of what the hardware can achieve? These are all real problems that had to be solved.\n\nAll of these challenges are difficult for humans and they cannot be one-shotted by an AI assistant. Where the AI can help is to do the “grunt work” of trying out different designs, testing hypothesis, and building tools and harnesses for other agents. AI works the best autonomously when either the problem space is narrow or the success criteria is wide. AI tends to fall on its face when you have a large problem space with a narrow success criteria. When I hear about “impressive” work done by AI assistant without a skilled operator, it always falls into one or more of the following groups:\n\n- It is a visually impressive demo but not actually technically challenging. (Wide success criteria)\n- It heavily relies on existing work. (Narrow problem space)\n- It is not useful or working beyond the demo. (Both)\n\nSo what is the value of the human operator? Their job is to break a large and intractable problem into a series of smaller ones that is friendly for an AI to solve… and more importantly for AI to validate. Don’t waste your time trying to prompt engineer or write skills or benchmark what the best coding harness is. All of that is table setting. The one skill you need to master is the same skill you needed before AI tools existed: the ability to break apart complex problems.\n\n## Forty days, on the record\n\nThe Triton bring-up started in late May, after the completion of the Neptune bring-up. There was a large break when Fable became unavailable because I had (correctly) hoped that it would be back soon (just not as soon as I had hoped).\n\n### The timeline\n\nEach marker is coloured by the model that did the work; a split marker means the milestone spanned more than one model. Counts on the chips are user turns.\n\n## The full bring-up, milestone by milestone33 entries · May 20 – Jul 23\n\n- May 20\n### VirtualBox teardown → Triton is designed\n\nA study of how VirtualBox runs D3D11 in a VM (DDI → SVGA3D command stream → hypervisor → host D3D11) ends with the decision to collapse the middle two layers:\n\n**Triton** becomes a user-mode library that turns Windows D3D11 DDI calls straight back into D3D11 API calls, which Neptune already forwards to the host.Opus 4.7**18** - May 20\n### DDI surface and the DXBC container\n\nThe DDI entry points get built out and inventoried. A first hallucinated\n\n`TritonDxbc`\n\nis thrown away and rewritten directly against VirtualBox’s`DevVGA-SVGA3d-dx-shader.cpp`\n\n— known-working code — with the chunking and command-encoding layers stripped out.Opus 4.7**32** - May 22–23\n### Windows KMD/UMD groundwork\n\nIn parallel on the Windows and desktop machines: viogpu3d build automation, an ARM64 build port, patch triage across three diverged driver forks, and a variant analysis of the async-path fixes.\n\nOpus 4.7**188** - May 24\n### Every DDI cross-checked against MSDN\n\nA full review pass: each DDI function looked up on MSDN, compared against both the docs and the VirtualBox implementation, written up in\n\n`DDI_REVIEW.md`\n\n, then every finding independently re-validated and fixed. Followed by a licensing, naming and comment cleanup.Opus 4.7**20** - May 25\n### Neptune wired into the Windows build\n\nvirtio-win-mesa (UMD) + viogpu3d (KMD) + neptune-protocol all building on the Windows dev VM, and a two-VM development loop stood up on the Linux box: dev VM builds, test VM runs under\n\n`-snapshot`\n\n, WinDbg over COM2, artifacts shared through virtiofs`Z:\\`\n\n.Opus 4.7**41** - May 26–28\n### Present path lands — and a TDR loop blocks the desktop\n\nThe DXGI present proposal is implemented on both sides and verified (\n\n`FlushToScreen scanout blob valid=1 1920x1080`\n\n, 54 clean swapchains). The desktop still doesn’t come up: a TDR → bugcheck → reboot loop is root-caused to an unregistered shader-resource view tearing down the context — a pre-existing DXVK shared-resource bug.Opus 4.7**41** Opus 4.8**6** - May 28 – Jul 1\n### Guest-side shared handles\n\nNeither DXVK nor the future backends can pass shared handles through, so they get emulated on the guest using the fence and context-blob machinery already in place. The session runs across the four-week gap; it ends by proving the live desktop capture works and cleanly\n\n*dis*proving app-liveness via draw-RT capture — DWM layers app windows through DirectComposition, invisible to the render server.Opus 4.8**71** - 6 days quiet\n- Jun 4–5\n### DXGI DDI study\n\nThe two virtualization drivers’ DXGI swapchain DDIs put side by side — Neptune’s WIP implementation against VirtualBox’s — to settle the architectural questions before the rework.\n\nOpus 4.7**17** Opus 4.8**3** - 5 days quiet\n- Jun 10\n### d3dmetal-native begun\n\n`libd3dmetal-native`\n\nstarts on the laptop: a native macOS library implementing the GFXT interface on top of Apple’s D3DMetal.framework — essentially dxvk-native but backed by Metal. It becomes the macOS host renderer that Triton talks to two weeks later.Fable 5**5** - 20 days quiet\n- Jul 1\n### Clean-slate re-bring-up\n\nEverything the previous agent had stashed stays stashed. A fresh context re-reads Neptune, Triton, KMD and UMD from scratch, reviews before writing, and rebuilds the whole host and guest stack. Part of that pass was confirming that\n\n**no VirtualBox code remains in the Triton driver**— VirtualBox stays a behavioural reference only, consulted for what a correct DDI implementation has to do (the “VBox-parity” DDI surface, and the find that`SetDisplayMode`\n\nwas a silent stub), never a source of code.Fable 5**8** - Jul 2\n*★*The Windows desktop renders**Live desktop up in the QEMU window**— user Active, presents flowing, zero errors. The last two crashes go with it: a KMD blob-info race (fixed with a locked snapshot + degenerate-scanout deferral) and a QEMU`qemu_memfd_alloc`\n\nabort, now structurally impossible via scanout-blob rect validation.Fable 5**8** - Jul 2–3\n### dmabuf replaces the fake shared handles\n\nThe emulated handles come out. DXVK gets real dmabuf import/export, and virglrenderer and guest Mesa drop the single-host-device / threaded-context sharing hack.\n\nMixedFable 5**18** Opus 4.8**7** - Jul 3\n### Artifact triage: glyphs, icons, compositing\n\nWith the desktop up, the rendering is wrong in interesting ways — wrong glyphs, half-noise icons, fully transparent windows with working title bars. The first job is a way to diagnose it without a human looking at screenshots.\n\nFable 5**23** Opus 4.8**4** - Jul 4\n*★*Fire Strike runs to completionA full 3DMark Fire Strike run finishes\n\n`EXIT 0`\n\nwith real scores (Graphics 1347 / Physics 1858 / Combined 369), rendering live to the GTK window — Demo and GT1 playing fullscreen with FlushToScreen past 900 presents.Opus 4.8**10** - Jul 5\n### Neptune on macOS: the cube spins\n\nThe other front: Neptune ported to macOS over d3dmetal-native, running under Wine + FEX in QEMU. First green run — 3,603 presents, zero crashes — after gating the Venus ring watchdog on CPUID emulator detection.\n\nOpus 4.8**26** - Jul 6\n### The perf ceiling is not the driver\n\nFire Strike GT1 is pinned at ~8 FPS and every driver-side lever moves it by nothing — which is the proof. GPU 99% busy at 300 MHz, package power 4 W of 50 W: a\n\n`30-amdgpu-pm.rules`\n\nudev clamp forcing`power_dpm_force_performance_level=low`\n\non every boot. The >5000 goal was unreachable, with receipts.Opus 4.8**28** Fable 5**5** - Jul 6\n### Stability: ten clean cycles\n\nTen consecutive boot → install driver → Fire Strike → screenshot → calc.exe → screenshot cycles, plus the fix for a long-standing wedge where the GTK window froze right after driver install.\n\nMixedOpus 4.8**9** Fable 5**4** - Jul 6–8\n### WSI moves to the guest; repos cleaned for upstream\n\nHost-side swapchain creation is removed entirely — the guest driver owns swapchain management and shares textures for scanout. virglrenderer, Mesa and DXVK histories are rewritten, diagnostics stripped, and the whole stack re-tested end to end.\n\nOpus 4.8**91** Fable 5**5** - Jul 9\n### Windows-on-ARM bring-up starts on Apple silicon\n\nThe desktop machine takes over: Windows on ARM64 under QEMU on macOS, Triton talking to d3dmetal-native on the host, KDNet debugging into a Windows dev machine over SSH. Nothing renders yet.\n\nMixedOpus 4.8**16** Fable 5**11** - Jul 10\n*★*Desktop, taskbar and GDI apps render on ARM64Five root fixes land: the DXBC converter SIGSEGV (ISGN scalar-mask widening), a KMD paging bugcheck, a GPU-residency wedge, vrend↔neptune shared-memory aliasing (which unblocked all window content), and host-side A8→RGBA8 translation for DWM’s glyph atlas. regedit is pixel-perfect; GPU-drawn glyphs are still wrong.\n\nMixedOpus 4.8**43** Fable 5**16** - Jul 12\n### Fire Strike perf pass on Apple silicon\n\nFire Strike made to launch, then profiled on both sides of the boundary — guest KMD/UMD and host render server — with symbols resolved, until the workload is GPU-bound.\n\nOpus 4.8**31** - Jul 13\n### A blank-frame detector in the host\n\nEvery few frames come back solid black or a flat color. Rather than eyeball screenshots, the host maps the first page of each frame and samples ~1024 pixels, so the bug can be reproduced and counted automatically.\n\nOpus 4.8**77** - Jul 14–16\n### The colour/RG-swap hunt\n\nFire Strike’s demo tints red or green and GT1’s particle effects come out the wrong colour — green flames. The hunt runs through Triton’s workaround code and D3DMetal-native.\n\nOpus 4.8**38** Fable 5**8** - Jul 15\n### DXBC container review finds the miscompile class\n\nA review of Triton’s DXBC container implementation against VirtualBox’s translator and the other reference implementations, aimed at the subtle shaper bug that makes the host translator miscompile some shaders.\n\nFable 5**5** - Jul 16\n### DXMT added as a second macOS backend\n\ndxmt-native gets fence and texture export/import over shmem and is wired into virglrenderer alongside d3dmetal-native — and unlike D3DMetal it builds native ARM64, no Rosetta. Linux + Wine validates first, then Triton + Windows.\n\nMixedOpus 4.8**15** Fable 5**11** - Jul 17\n*★*Fire Strike to completion on refactored DXMTThe refactored DXMT renders Fire Strike to completion at\n\n**4989**(Graphics 6213 / Physics 5068 / Combined 1995) on the fast host — confirmed against the exact HEAD build, with the earlier “renders nothing” shown to be intermittent rather than a break in the refactor.MixedOpus 4.8**14** Fable 5**9** - Jul 17\n### arm64x packaging\n\nThe Triton UMD is packaged as arm64x — arm64ec and x64 in one binary — through a dedicated build step rather than out-of-band scripts, the KMD gains a user service, and a full build-and-test cycle runs against both host backends.\n\nOpus 4.8**72** - Jul 18\n### DWM layer drops traced to the impostor texture\n\nWindowed apps lose whole composition layers on D3DMetal-native — empty start menus, missing search boxes, borderless windows. The trail leads into\n\n`substitute()`\n\n, where shmem-backed impostor textures stand in for shared textures so they can cross process boundaries.Opus 4.8**42** - Jul 19\n### Device-lost and black-screen hangs fixed\n\nTwo black-screen variants — immediately after driver install, and after a complete Fire Strike session — chased through KDNet, guest and host tooling; plus an audit of the impostor-texture mechanism against Apple’s docs.\n\nOpus 4.8**99** - Jul 20\n### History collapsed to one Triton commit\n\nUpstream prep: every follow-up commit that belongs to Triton is rolled into a single introducing commit, with Neptune-side and unrelated changes kept separate. Performance characterisation of the D3DMetal-native path begins.\n\nOpus 4.8**57** - Jul 21–22\n### Present-gate pipelining\n\nPresent was blocking on GPU work, leaving both CPU and GPU under-utilised. Unblocking it let presents race ahead into black frames; the fix moves the gate out of the UMD DDI and onto the KMD mechanism designed for it.\n\nOpus 4.8**70** Kimi K3**16** Sonnet 5**7** - Jul 23\n*★*Final numbersBack-to-back clean Fire Strike runs on the Windows guest:\n\n**5124** on DXMT (ARM64) and**5682** on D3DMetal (x86_64/Rosetta), all four workloads,`EXITCODE=0`\n\n, black-frame rate ≤0.8%. The Linux/Wine reference on the same host scores**7624**.Fable 5** 54**Opus 4.8** 6** - Jul 23\n### Upstream review pass over UMD and KMD\n\nInteractive, item-by-item review of the guest driver code — development narratives out of the comments, bring-up log entries accounted for, MSDN contracts re-checked — ahead of publication.\n\nFable 5**16**\n\n### User turns per day\n\n**1418** Opus 4.7\n\n**424** Fable 5\n\n**275** Kimi K3\n\n**16** Sonnet 5\n\n**7** No reply\n\n**38**\n\n## Table view — turns per day by model\n\n| Day | Opus 4.8 | Opus 4.7 | Fable 5 | Kimi K3 | Sonnet 5 | No reply | Turns | Est. cost |\n|---|---|---|---|---|---|---|---|---|\n| 2026-05-20 | 62 | 62 | $225.98 | |||||\n| 2026-05-21 | 7 | 5 | 12 | $2.18 | ||||\n| 2026-05-22 | 167 | 167 | $330.36 | |||||\n| 2026-05-23 | 43 | 43 | $204.83 | |||||\n| 2026-05-24 | 26 | 26 | $66.77 | |||||\n| 2026-05-25 | 37 | 37 | $224.60 | |||||\n| 2026-05-26 | 23 | 23 | $404.88 | |||||\n| 2026-05-27 | 31 | 1 | 32 | $298.21 | ||||\n| 2026-05-28 | 12 | 6 | 18 | $266.97 | ||||\n| 2026-05-29 | 30 | 30 | $573.48 | |||||\n| 2026-05-30 | 8 | 1 | 9 | $53.71 | ||||\n| 2026-06-04 | 3 | 7 | 10 | $4.96 | ||||\n| 2026-06-05 | 10 | 10 | $4.70 | |||||\n| 2026-06-10 | 5 | 1 | 6 | $85.90 | ||||\n| 2026-06-28 | 14 | 14 | $272.97 | |||||\n| 2026-06-29 | 11 | 11 | $351.75 | |||||\n| 2026-06-30 | 8 | 8 | $192.34 | |||||\n| 2026-07-01 | 15 | 5 | 13 | 33 | $486.88 | |||\n| 2026-07-02 | 24 | 28 | 52 | $553.05 | ||||\n| 2026-07-03 | 10 | 55 | 65 | $448.39 | ||||\n| 2026-07-04 | 25 | 25 | 50 | $876.18 | ||||\n| 2026-07-05 | 58 | 1 | 59 | $471.15 | ||||\n| 2026-07-06 | 133 | 13 | 146 | $599.92 | ||||\n| 2026-07-07 | 55 | 1 | 56 | $407.79 | ||||\n| 2026-07-08 | 59 | 3 | 62 | $107.37 | ||||\n| 2026-07-09 | 53 | 1 | 29 | 3 | 86 | $664.24 | ||\n| 2026-07-10 | 3 | 11 | 14 | $597.21 | ||||\n| 2026-07-11 | 11 | 3 | 14 | $524.09 | ||||\n| 2026-07-12 | 97 | 2 | 99 | $483.60 | ||||\n| 2026-07-13 | 62 | 62 | $421.32 | |||||\n| 2026-07-14 | 46 | 3 | 49 | $377.41 | ||||\n| 2026-07-15 | 31 | 2 | 33 | $261.41 | ||||\n| 2026-07-16 | 5 | 22 | 27 | $676.39 | ||||\n| 2026-07-17 | 153 | 6 | 159 | $438.89 | ||||\n| 2026-07-18 | 133 | 5 | 138 | $524.43 | ||||\n| 2026-07-19 | 137 | 2 | 139 | $513.46 | ||||\n| 2026-07-20 | 97 | 2 | 99 | $542.57 | ||||\n| 2026-07-21 | 92 | 92 | $568.66 | |||||\n| 2026-07-22 | 10 | 16 | 7 | 33 | $113.46 | |||\n| 2026-07-23 | 23 | 70 | 93 | $537.50 | ||||\n| Total | 1418 | 424 | 275 | 16 | 7 | 38 | 2178 | $14,760 |\n\nTwo things fall out of that chart. The bring-up is not a smooth ramp — it is four bursts separated by two multi-week gaps, and the busiest days are all in the last fortnight, after the driver already worked. And the 2,178 prompts are spread over just 40 active days, which is roughly 54 typed messages on a day when anything was happening at all.\n\n### Daily token usage\n\n### Model usage\n\n- Opus 4.848.2M67.3%\n- Opus 4.711.1M15.5%\n- Fable 511M15.4%\n- Haiku 4.5353K0.5%\n- Kimi K3926.7K1.3%\n- Sonnet 52.66K<0.1%\n\n## Table view — per model\n\n| Model | User turns | API turns | Input | Output | Cache read | Cache write | Est. cost |\n|---|---|---|---|---|---|---|---|\n| Opus 4.8 | 1418 | 31,683 | 2.02M | 46.2M | 11.5B | 152.7M | $7,887 |\n| Opus 4.7 | 424 | 8,054 | 307.3K | 10.8M | 2.63B | 40.9M | $1,840 |\n| Fable 5 | 275 | 10,531 | 1.07M | 9.94M | 4.05B | 37.3M | $5,018 |\n| Haiku 4.5 | 0 | 1,383 | 18.9K | 334.1K | 64.5M | 2.59M | $11.37 |\n| Kimi K3 | 16 | 274 | 763.4K | 163.3K | 70.5M | 0 | n/a |\n| Sonnet 5 | 7 | 12 | 24 | 2.64K | 5M | 425.7K | $3.14 |\n\n## Where does the human fit in?\n\nMy subjective feelings were that I had just as much cognitive load–if not more–than writing code without any AI. The shape of that load was different though. When writing code, the difficulty stems from trying to keep track of all the moving pieces in my head and trying to convert these abstract ideas into concrete code. The most rewarding feeling is seeing the code work after hours or days of coding. For this project, I have not written a single line of code myself. However it is still the same feeling of having these abstract ideas and trying to guide the AI into producing something that *feels* correct. A lot of the work is in reading the code and in debugging issues (just like before). Once a part is complete and I read the code and it looks *good* and I run it and it *works*, I get the same feeling of satisfaction.\n\n### The shape of it · One person, 59,443 words, 45,893 tool calls\n\nOver nine weeks of bringing up a Direct3D 11 driver stack across QEMU, virglrenderer, Mesa, a Windows WDDM kernel driver and a Metal backend, the human typed **1,576 messages totalling 59,443 words** — about 120 paperback pages. The models answered with **101,031 messages**, 45,893 tool calls, 67.4M output tokens and roughly **1.84 million words** of prose.\n\nThat is a 31:1 ratio of model words to human words. Put differently: **every word the human typed bought about 0.77 tool calls**, and every hour of model work cost about 125 human words.\n\nThis is the clearest evidence for what I said at the beginning: the tools allow you to amplify your existing skills. In this case, ~60 hours of human work amounted to ~480 hours of machine work.\n\n### Finding 1 · It is not a continue button\n\nThe cheapest story about AI-assisted engineering is that the human degenerates into a keystroke: *continue, go ahead, yes, ship it.* The transcripts say otherwise, and they say it precisely — because “continue” is a short, exact string that can be counted.\n\nOf 1,576 human messages, **138 carried no new information**: 88 pure continuations (`continue`\n\n, `yes`\n\n, `go ahead`\n\n, `keep going`\n\n), 36 status polls (`ping`\n\n, `any updates?`\n\n, `is it done?`\n\n) and 14 arbitrations where the human picked among options the model had laid out (`try option 1`\n\n, `let's do B1+B2`\n\n). That is **8.8%**. The remaining **91.2% carried content**, and they carried 99.5% of the words.\n\nThere is a twist worth noticing. When the human *did* just say “continue,” it bought **more** autonomous work than an average substantive message — a median of 15 tool calls against 7. “Continue” is not a filler turn; it is the specific instruction *keep grinding on the thing you already understand*, and it is issued precisely when the model already has enough context to grind.\n\n### Finding 2 · Zero messages about how to use a tool\n\nAcross 1,576 human messages there is not one instruction about search or file mechanics. No “grep for”, no “use ripgrep”, no “cat that file”, no “open the header”. Zero.\n\nMeanwhile the models made 29,081 Bash calls, 6,293 edits, 5,968 reads and 1,048 writes. The entire mechanical layer of the work — deciding what to search, which VM to SSH into, which build script to run, where the log went — was never once specified.\n\nWhat the human *does* specify is one level up, and it is remarkably consistent:\n\n| The human’s actual vocabulary | Messages | Share |\n|---|---|---|\n| Cites a specific git commit SHA (207 distinct SHAs) | 135 | 8.6% |\n| Names a specific source file | 129 | 8.2% |\n| Invokes MSDN / a spec / an API contract as ground truth | 66 | 4.2% |\n| States a standing rule the model should carry forward | 97 | 6.2% |\n| Tells the model how to search, read, or grep | 0 | 0% |\n\nThis is what “focusing on the big picture” looks like when you go measure it. The human’s register is commits, DDI entry points, render encoders, MSDN contracts and architectural trade-offs. The register of tool calls, file paths and shell invocations simply never appears — not because the human suppressed it, but because it never came up.\n\n“One level up” is a good way to put it. I have been thinking about how in the early days, people programmed in machine code. Then assembly code was invented to abstract away the encoding of instructions so human can focus on the semantics. Then higher level languages like C and Python were invented so humans can focus on the logical structure rather than the semantics. Now, with AI assistants, we can focus on the design and architecture rather than the logical structures.\n\n### Finding 3 · The model explains; the human decides\n\n**36.5% of substantive messages are questions or requests for explanation** — 525 of 1,438. And these turns behave completely differently from the rest of the corpus.\n\nQuestions are cheap for the model and expensive in value for the human. They cost a couple of minutes, they mostly don’t touch code, and **56% of the time the human’s very next message is a directive** — the explanation gets converted straight into an instruction. The pattern is visible turn by turn: `ask → act`\n\nis 15.1% of all turn-to-turn transitions, almost exactly balanced against `act → ask`\n\nat 15.7%.\n\nThe mechanism is worth naming, because it is the actual source of the human’s leverage. The human does not need to have read the 400,000 lines of VirtualBox, dxgkrnl, DXVK and D3DMetal in this project. They need a model that can read them on demand and explain the call chain — and then their own domain judgement is enough to say *that’s wrong, DXGI should already be limiting inflight frames, check the MSDN contract.* Explanation is what converts one person’s judgement into coverage over a codebase they could not otherwise hold in their head.\n\nI have found that AI models are much, *much*, better at understanding code than writing new code. Even if you are against using AI tools to write code (and that is an understandable position given how much work it is to properly review the code), you should be using it to understand new codebases.\n\n### Finding 4 · Reading the generated code is a job, and it has a measurable footprint\n\nFourteen sessions open with an explicit protocol the human wrote, some version of: *“We will be doing an interactive code review. Each review item is either a question or a request for change. If it is a question, do not jump to changing the code.”*\n\nThose 14 sessions contain **183 human messages against 2,626 tool calls — 14.3 tool calls per human turn**. Everywhere else in the corpus the figure is **31.1**. Review mode is more than twice as human-dense as the rest of the work, and it eats about 25 engaged hours.\n\nThe extreme case is a rebase review on 2026‑05‑22: **92 human messages, 566 tool calls — 6.2 tool calls per turn.** That is a person going through a commit history line by line with a machine doing the typing.\n\nWhat review turns up is not stylistic. A sample from one three-hour session on virglrenderer:\n\n`npt_context_import_resource`\n\nwe removed the code for `close(fd)`\n\nthen set `res->u.fd = fd`\n\n. When we have `VIRGL_RESOURCE_FD_SHM`\n\ndoes this not leak `res->u.data`\n\nin the union? this change was added in context of d3dmetal-native but is this change safe for all backends including dmabuf on linux?”`npt_cs.h`\n\nthat is just an empty line. drop that change. also remember for the final pass to do the same in other commits as well.”And the corrections land. In 1,576 messages there are only **24 cases of the human having to repeat themselves** to consecutive turns — 1.5%. Pointing at the problem is usually sufficient; the model does not generally need to be told twice what the fix is.\n\nWhat it *does* need is emphasis. **30.1% of human messages contain shouted words** — ALWAYS, NOT, ONLY, NEVER, DO NOT — and the targets are consistent: don’t guess, don’t hand-wave, don’t push, don’t commit diagnostics, don’t stop.\n\n### Finding 5 · The expensive problem is that the model cannot see\n\nThis project’s hardest bugs were visual: a black screen, a missing search pill in the Windows taskbar, glyphs that don’t render, a magenta tint over an aurora, frames that flash black at 30 Hz. The model has no eyes. For nine weeks, **the human was the display.**\n\n**139 messages** are the human reporting what is on a screen the model cannot query. Read in date order they read like an instrument log, and the language sharpens over time:\n\n**May 28**— “I do see the desktop after the driver load but it seems to have loaded one frame and the entire process is frozen”\n\n**Jul 4**— “I see the calculator now but as I move the mouse cursor around, there are random artifacts that appear… once the mouse stops, after a few seconds, it refreshes and looks right again”\n\n**Jul 17**— “I no longer notice the hangs but now we have a constant micro-stuttering. As GT1 runs, I constantly see flashes of black screen that feels like 1-2 frames of black. It is persistent through the entire run.”\n\n**Jul 23**— “I see a completely black screen for 17 seconds. Then I see frame 801 stuck for the next few seconds. Then it terminates.”\n\nThat progression — from “frozen” to “1–2 frames of black” to “17 seconds, then frame 801” — *is* the work. The human is converting a perception into a quantity the model can go look for. Until that conversion happens, the model is guessing.\n\nThe transcripts contain the moment where this becomes explicit, and the human is blunt about it twice:\n\nThe measurable version\n\nA message reporting a **visual symptom with no instruction about how to observe it** buys a median of **8 tool calls**. A message that tells the model **what to instrument, dump, trace or capture** buys **18** — and runs 11.2 minutes instead of 7.4. Same human, same bugs, same models. The difference is whether the model has been handed a way to see.\n\nI do feel this is a problem that will be solved with better models.\n\n#### Case study: the missing search pill\n\nOn 2026‑07‑18 the human opened a session with a description of a rendering bug that had persisted “since day 1”: under the D3DMetal backend, the Windows desktop drops compositing layers. The Start menu is sometimes an empty box; the taskbar’s rounded search box renders as bare text on the bar. Twenty-seven human messages and 705 tool calls later, over fourteen hours, the shape of the human’s day is this:\n\nThe arc is the whole thesis in one day. It opens with the symptom described in words. The model immediately starts bisecting compiler workaround flags, and the human has to stop it — three times, twice by interrupt:\n\nThen eighteen turns of hypothesis and small experiments — the 2DTextureArray view, `CheckFormatSupport`\n\n, whether DWM ever queries `D3D11_FEATURE_FORMAT_SUPPORT2`\n\n— mostly cheap runs of 2 to 40 tool calls. At turn 18 the human asks for the first real data channel: *“can you dump the DXBC → Metal shaders for each DXBC. try to find the one that composites the search pill. then compare the metal code for both.”* At turn 19 they escalate it to *“get the AIR of both sides and compare them”* — and the model runs 191 tool calls, then the human disappears for six and a half hours.\n\nAnd then the ending, which is the part worth sitting with. The human comes back having done the observation work themselves:\n\n`dwm-render-debug`\n\n. I’ve dumped the PNG for each render encoder color 0 attachment output at the end of each render encoder. The last image shows the rendered taskbar. In DXMT, the taskbar is complete: clock, icons, search pill. In D3DM, it is missing many of those elements.”Followed by pasting a raw Metal command stack into the chat — *“do you see the missing draws”* — and then, two turns later, explaining the file format to the model: *“each empty line separates a render encoder.”*\n\nThe bottleneck was never the model’s reasoning. It was the absence of a channel between a pixel on a screen and a token the model could read. Building that channel took a human most of a day.\n\nThere is a second-order effect worth flagging: **48% of a session’s human messages fall in the first third of its span.** The human’s work is front-loaded, and the session-opening brief is the highest-leverage thing they write — a median 52 words against 18 for later turns, buying 17 median tool calls against 8. Roughly half the messages in a session are spent aiming it.\n\n**Also: I cannot sit here and watch the whole thing.**”\n\nOnce I was able to get the AI to empirically measure the failure and set a goal to fix that metric, the hard work is “done” and the rest can be automated.\n\n### Finding 6 · The human compiled their own continue button\n\nBecause they could not sit there and watch, they wrote down what “done” means and made the harness enforce it. Across the corpus there are **49 distinct /goal conditions** — success predicates registered as stop-hooks — that fired\n\n**395 times**. Of those,\n\n**367 fires blocked the model from ending its turn.**\n\nThree hundred and sixty-seven times, the model tried to stop and a sentence the human had written earlier pushed it back in. That is the “continue” key, turned into a spec.\n\n| Goal condition (abridged) | Fires | Blocked |\n|---|---|---|\n| “Figure out the difference between DXMT and D3DMetal in pill rendering… Keep going until you figure out why the pill is not rendered, confirm it, patch in a fix, and test that it works.” | 56 | 56 |\n| “Fix all the P1 items. Make sure to follow the same pattern: validate API calls with MSDN documentation, cross-check against virtualbox, and commit without narrative comments” | 48 | 48 |\n| “Fix all three issues… Make sure the VM can run for 15 minutes without crashing or going out of memory while maintaining visible scanouts.” | 35 | 35 |\n| “fix the black screen” | 35 | 34 |\n| “Achieve > 5000 on 3dmark firestrike” | 23 | 23 |\n\nWhat is striking is not the goals that name an outcome. It is how many of them name a *method*. The human is not only specifying what done looks like — they are specifying the epistemics:\n\nAnd separately, typed into the conversation as a standing instruction:\n\n**do not prompt me for guidance unless you cannot follow these rules.**”\n\nThat last clause is the human explicitly pricing their own attention.\n\n### Finding 7 · Less hand-holding, quantified\n\nI felt that Fable 5 required a lot less babysitting than Opus 4.8. Here is the data to back it up.\n\nThe corpus spans three model generations on overlapping work, which makes “this one needed less babysitting” a testable claim rather than an impression. It holds.\n\n| Model | Runs | Median tools | Mean | p90 | Median run | Interrupted |\n|---|---|---|---|---|---|---|\n| Fable 5 | 222 | 17 | 53.8 | 152 | 9.1 min | 14.4% |\n| Opus 4.8 | 1,109 | 8 | 24.0 | 63 | 5.1 min | 22.3% |\n| Opus 4.7 | 253 | 6 | 27.5 | 92 | 2.5 min | 12.6% |\n\nModel choice was not randomised, so the honest check is within-project, where the task mix is at least comparable:\n\n| Repository | Fable 5 median | Opus 4.8 median |\n|---|---|---|\n| Neptune-QEMU | 21 (n=94) | 9 (n=445) |\n| VMs (driver bring-up) | 16 (n=48) | 11 (n=220) |\n| kvm-guest-drivers-windows | 19 (n=7) | 7 (n=50) |\n| dxmt | 38 (n=4) | 5 (n=17) |\n| d3dmetal-native | 6 (n=49) | 14 (n=31) |\n\nFour of five go the same way, one goes the other. The two longest unattended runs in the entire corpus are both Fable: **554 tool calls over 6 hours** and **541 tool calls over 13.5 hours**, each launched by a single human message.\n\nThe “but I still have to step in” half is equally visible: **one Fable run in seven was interrupted by the human hitting escape**. Across all models it is 19.0% — 321 runs, 334 interrupt events. Roughly one intervention every five turns, all corpus long, at every model generation.\n\n### Finding 8 · The human got more involved over time, not less\n\nThis is the result that runs against the intuition, and it is not an artefact of session count — it survives normalising by engaged hours.\n\nOver the same period the mix of what the human was saying shifted decisively. The share of messages carrying a data-collection instruction went from **9% to 24%**. Tool calls bought per human message fell from 39 to 24.\n\nThe explanation is in the nature of the work, not the quality of the models. Late May was greenfield: *make anything appear on the screen.* That is a goal you can state in one sentence and walk away from — and the human did, at 1.86 messages per hour. Late July was *make the taskbar composite correctly, eliminate the 40% black-frame rate, get Present off the critical path, and clean the commit history for upstream.* Those are goals where the human is the oracle for whether it worked, the reviewer of every line that ships, and the designer of the measurement.\n\nThe pattern\n\nAutonomy is highest when the goal is coarse and failure is obvious. It drops exactly where the work gets valuable — where “correct” is a judgement call, where the evidence is a pixel, and where the code has to be good enough for someone else to merge.\n\nThis makes sense because the initial bring-up phase was mostly just guiding the AI into a good design. The debug (correctness and performance) phase required a lot more intervention due to the previously stated difficulties.\n\n### So, where? · The human is the part that cannot be delegated because it touches the world\n\nReading nine weeks of this backwards, the human’s residual job sorts into four things, and none of them is writing code:\n\n**Perception.** The model cannot see the screen, hear the fan, or feel that the cursor lags. 139 messages exist purely to carry a perception across that gap, and the hardest single day in the corpus was spent building a pipe so the model could see one taskbar.**Judgement about what “correct” means.** 49 goal predicates, 97 standing rules, 66 appeals to a spec, and an interactive review protocol the human wrote and ran fourteen times. The model can check a program against a rule; someone has to author the rule.**Decomposition and altitude.** The session-opening brief is the single highest-leverage artefact in the corpus — 3× longer than an average message and buying 2× the autonomous work. Zero of 1,576 messages descend to tool mechanics.**Being the world’s actuator.** Rebooting the host, restarting the VM, and telling the model what happened.\n\nThe volumes are worth restating plainly. The human spent about **65 engaged hours** on the loop across nine weeks and produced **59,443 words**. The models spent **476 hours** and produced 1.84 million words plus 45,893 tool calls. The leverage is roughly 7:1 in time and 31:1 in output.\n\nBut the leverage is not free and it is not increasing. Over the nine weeks the human’s rate of input *doubled* — because the work moved from “make it render” to “make it render correctly,” and correctness is exactly the region where the model needs a person to be the eyes, the spec, and the reviewer. The bottleneck did not disappear. It migrated, from typing to seeing.\n\n## Appendix: one bug, three models\n\nOne entry in that timeline supports a comparison the rest of the project does not. Present-gate pipelining — the second-to-last milestone, 21–23 July — was worked by three different models on three consecutive days, two of them from a verbatim-identical opening prompt. The next part compares them at equal token spend. It is not a benchmark, and the fairness ledger below matters more than the headline; but it is a rare look at what “needs less hand-holding” means when it is counted rather than felt.\n\nWhen Kimi K3 came out, many people online claimed it was as good as Fable 5 so I was curious to test the claim out. This analysis is not a fair comparison of capability because I limited my Moonshot token spend to $30 but I think what the model did with that $30 is an interesting story to read.\n\n### The three sessions\n\nAll three ran on the same machine against, driving the same Windows-on-aarch64 guest through the same `nept.py`\n\nharness.\n\n| Opus 4.8 | Kimi K3 | Fable 5 | |\n|---|---|---|---|\n| Session | `a0816380` | `d8f976e3` |\n`ff50fe7a` → `daff6027` |\n| Model phase | Jul 21 11:05 → Jul 22 09:58 | Jul 22 11:58 → 15:11 | Jul 23 00:20 → 18:14 |\n| Wall clock | 22.9 h | 3.21 h | 17.9 h |\n| Active time (gaps ≤5 min) | 13.9 h | 2.90 h | 5.71 h |\n| API responses | 1,128 | 274 | 611 |\n| Tool calls | 1,101 | 301 | 599 |\n| Output tokens | 1,798,997 | 163,274 | 612,249 |\n| Billed tokens | 533.1 M | 71.4 M | 322.0 M |\n| Spend | $339 list | $30 hard cap |\n$392 list |\n| Ended because | user asked for cleanup + handoff | `429 · insufficient balance` |\nuser exited after a regression check |\n\n**71.4 M billed tokens**— 763 K uncached input, 163 K output, 70.5 M cache reads, zero cache writes. That number is the yardstick for everything below.\n\n### The problem all three inherited\n\nA paravirtual WDDM driver stack: guest UMD (`virtio-win-mesa`\n\n, the Triton D3D11 driver) → KMD (`kvm-guest-drivers-windows`\n\n) → virtio → host `virglrenderer`\n\n→ QEMU cocoa → Metal. Present was blocking the app thread while the GPU worked, so CPU and GPU never overlapped: 3DMark Fire Strike GT1 ran at ~27 fps with the render thread parked inside `Present`\n\n.\n\nRemoving the block unlocked pipelining and broke correctness. The user’s opening prompt — *identical, verbatim, to Kimi and to Fable* — named three failure modes and told the model to distrust every prior conclusion:\n\n*“…audit the code at HEAD (KMD+UMD) focusing on the events, fencing, and present\narchitecture… Then build and test the driver and try to detect the different failure modes\nwe’ve observed: 1) flashing black frames (likely inert fences → present before frame is\nready), 2) stuck frames (likely stuck fences → present gets blocked in the pipe), and\n3) underutilized GPU (due to lack of pipelining). The existing test and diagnostics the\nlast agent produced is incomplete.”*\n\nOpus 4.8’s prompt the previous day was different in form — it began\nfrom `HANDOFF-perf-analysis-methodology.md`\n\nand asked for triage and a diagnostic\nA/B path rather than an audit of a specific HEAD.\n\n### Was Kimi K3 going in the right direction?\n\nMostly yes. Its progress report at 13:43 made ten checkable claims. Six hold up against everything measured afterwards, two are correct for the exact tree it audited, and two are wrong — one of them badly, because it closed off the failure mode the user had explicitly asked about.\n\n| Kimi’s claim | Verdict | Adjudicated by |\n|---|---|---|\nFailure mode 3 (GPU underutilisation) is already fixed at HEAD — overlap index ≈1.1, GT1 50.2 fps DXMT / 77.6 D3DMetal vs the 27.2 fps blocking baseline |\nRight | Fable measured overlap ≈1.0 on the same HEAD independently the next morning |\nThe broken piece is the association between a fence arm and the flip it gates — flips run un-gated (`tok=0` ) → black/torn frames |\nRight | Fable’s independent audit reached the same conclusion and proved it live: `NPT-DIAG promote tok=0 done=145 zero=129 nonzero=0` . The fix that finally shipped, `5d984dc3` , is a token-association fix |\nThe mechanism is the 16-slot pid-keyed token table exhausting after 16 presenter pids, then dropping silently |\nRight for its tree | True of HEAD + the 736 uncommitted lines Kimi audited. At clean HEAD the mechanism is different — the escape lands on the npt transport’s own private `D3DKMT` device, so every flip reads 0 from the first frame. Both are correct; they audited different trees |\nA per-device stamp via a user-mode : `D3DKMT` handle is impossible`VioGpuDevice::FromHandle` calls a virtual `Magic()` on a foreign pointer, and `DXGK_HANDLE_TYPE` only has ALLOCATION and RESOURCE |\nRight | Kimi built it, bugchecked the guest with 0xFC, and diagnosed why. Recorded as do-not-retry and never revisited by anyone. This is its most durable contribution |\nThe fix is one adapter-global monotonic token, peeked by the flip at latch time |\nAdopted, then killed | Fable shipped precisely this as `b2b04618` , then proved it livelocks a pipelined app — at depth 3 with steady frame times flip N+1 always latches before token N retires, so each new latch replaces the almost-ready one. Reverted, then dropped from history |\nIts own follow-on redesign: contiguous-prefix retirement, so `done ≥ T` implies every token ≤ T retired |\nUnsound | Completions aren’t 1:1 with arms in token order (`PFENCE-ARM #5632 token=5632` vs `PFENCE-CB #5632 token=5730` , `wrap=72` ), so the walk wedges permanently — `done` frozen at 651 while flips asked for 2822. Caught by Opus 4.8 1h58m after Kimi’s budget died; Kimi never compiled it |\nStuck frames are not stuck fences — the 2–4 s mid-GT1 gaps are app-side guest CPU |\nRight | Fable’s 51-second guest ETW trace: the frame-producing thread is 92–100% on-CPU straight through the hitch window, waits ≤8%, disk utilisation ≤30% — single-threaded asset ingest inside the statically-linked exe |\nThe 30–60 s freezes are an idle desktop with no presents — benign |\nWrong | The multi-second frozen frame was real and had two driver causes, both fixed later: FlipThread vsync starvation (every fence-completion wake restarted the vsync period, and MMIO-flip completion is reported only by the CRTC_VSYNC interrupt — a 1123 ms present stall was measured), and unbounded CPU run-ahead (every token in the flip queue seconds from retiring → 5–15 scanouts/s). Fable found the vsync bug in its first 46 minutes |\nThe `VIOGPU_GET_GATE_STATS` escape the old probe expects no longer exists in the KMD |\nRight | Fable later re-implemented it as a diagnostic patch |\nTwo stale claims corrected: the KMD comment “the current UMD passes NULL” is false; `c88154be` 's “1 s bounded fallback” is 10 s in code |\nRight | Both carried into the handoff document unchallenged |\n| Ten claims | 6 ✓ · 2 ◐ · 2 ✗ |\n\n### Was Kimi efficient with those 71.4 M tokens?\n\nStructurally, yes — it was the *leanest* of the three per unit of work attempted. Behaviourally, no — a large share of the budget went into recovering from its own malformed tool calls and into thinking rather than acting.\n\n#### Tool-call failure rate\n\nEvery failed tool call costs a full context replay to recover from. Kimi’s rate was roughly double Fable’s and an order of magnitude above Opus’s at the same point in their sessions. “Soft failures” include exit code 1, `String to replace not found`\n\n, `Build FAILED`\n\nand empty-result greps.\n\n- Kimi — hard errors8.0%\n- Kimi — incl. soft12.0%\n- Fable — hard errors4.3%\n- Fable — incl. soft6.9%\n- Opus — hard errors0.9%\n- Opus — incl. soft10.8%\n\n**Bash 10.3%**/\n\n**Edit 13.6%**; Fable\n\n**Bash 5.6%**/\n\n**Edit 0%**; Opus\n\n**Bash 0.5%**/\n\n**Edit 5.0%**. Opus’s comparatively high soft rate is mostly deliberate probing — greps and\n\n`findstr`\n\nqueries that legitimately return nothing.#### Where the 3h13m actually went\n\nInside those blocks, four concrete friction episodes ate roughly 40 minutes — about a fifth of the whole session — and the user eventually asked about it directly:\n\n**CRLF versus** Kimi tried to edit`Edit`\n\nanchors — ~16 min.`viogpum.h`\n\nfrom a reconstructed anchor string; the file is CRLF, so the exact match failed. It wrote an anchor-asserting Python script, that anchor didn’t match either, and it ended up inspecting the bytes with`file`\n\n,`cat -A`\n\nand`repr()`\n\n. Its own thinking later names the cause exactly:*“my anchors keep being wrong because I’m reconstructing file contents from memory/diffs.”***Fetching one WDK header — 10 turns, ~8 min.** Looking for`d3dkmddi.h`\n\non the Windows dev box: wrong include subdirectory (`km/`\n\ninstead of`shared/`\n\n), then five different shell quotings of the same`scp`\n\n, before giving up and`copy`\n\n-ing it to the home directory first.**Eleven consecutive greps** chasing`SetEventOnCompletion`\n\nthrough virglrenderer, most returning nothing, each one a full context replay.**Seven turns to launch a sampler**— discovering that`start /min`\n\ndoesn’t detach over SSH, then that`nept.py test`\n\nblocks.**The user eventually asked about it directly — and Kimi took the point.** At 15:03, mid-turn, the user typed:*“Why do you keep fighting with tools calls? Why not just scp the files locally and then scp it back?”*Two minutes later Kimi’s thinking answers it:*“That’s… exactly what I’m doing? The friction was (1) CRLF line endings breaking exact-string Edit matches, (2) my guessed anchor text not matching the actual file… Better: read the actual local file first, then edit with exact text.”*It changed method and the edits stopped failing. That is a mid-turn steer absorbed correctly without a restart — the one piece of technical direction it got, and it landed.\n\nTwo failures were not Kimi’s fault and should be discounted from any judgement of the model:\n\n**13 minutes dead to a protocol incompatibility.** Between 14:16 and 14:29 the endpoint returned`400 Invalid request: the message at position 512 with role 'assistant' must not be empty`\n\nfive times. The transcript shows the user cycling`/exit`\n\n→`continue`\n\nfive times to get past it. Six of the nine “continue” prompts in this session are that recovery loop, not the model stalling.**The budget ended the session mid-edit.** First`429 · account suspended due to insufficient balance`\n\nat 14:44, terminal at 15:14. Kimi’s last action was an`Edit`\n\nat 15:11:11 fixing a`C2065`\n\nforward-reference it had just correctly diagnosed. It never got to compile, let alone test, its own redesign.\n\n**Efficiency verdict.** Per round trip Kimi was cheap — smallest context, fewest tokens per tool call. What it lacked was\n\n*yield*: 596 output tokens per response of which roughly two-thirds was internal reasoning, 7 K characters of prose to the user in three hours, and a tool-failure rate that turned a meaningful slice of the budget into recovery. It was not profligate; it was slow to convert tokens into verified state.\n\nThis was my subjective experience as well. Kimi K3 spent most of its tokens fighting the tooling but when it got past the hurdle, the analysis it made felt smarter than Opus 4.8.\n\n### Fable 5 on the same prompt, at fractions of the same budget\n\nThe interesting numbers aren’t at the iso-token line — they’re well before it.\n\n| Milestone | Wall clock | Responses | Billed | % of Kimi’s budget | List cost |\n|---|---|---|---|---|---|\n| Full audit delivered — two root causes, both proven live with its own instrumentation, MSDN contract points verified, diagnostics patch + README written, memory updated | 46 min | 78 | 14.4 M | 20% | $22 |\nBoth fixes implemented, built, committed, and validated — GT1 ×3 at 47.2/46.2/48.7 fps, tear test 0/20, overlap ≈1.0, `timeouts=0` . Then it stopped and idled for five hours. |\n1 h 57 m | 197 | 57.1 M | 80% | $73 |\nIso-token line — by now three hours deep into a second problem (fullscreen scanout going headless) with the user watching the screen |\n7 h 20 m | 228 | 71.5 M | 100% | $94 |\n| Session end — UMD run-ahead pacing, QEMU cocoa layer fixes, virglrenderer fence publication, an ETW investigation that falsified its own theory, a full Fire Strike matrix on both backends, and a history rewrite | 17 h 54 m | 611 | 322.0 M | 451% | $392 |\n\n**20% of Kimi’s token budget**, and a committed, benchmark-validated fix on\n\n**80%**. Then it went idle for five hours because it had finished what it was asked to do — which is itself a signal worth noting next to Kimi’s nine “continue” prompts.\n\n#### The fairness ledger\n\nFour things separate these two runs besides the model. Two favour Fable heavily.\n\n**Fable started with Kimi’s conclusion in its context.** This is the big one. Claude Code auto-loads`MEMORY.md`\n\nat session start, and the top line at 00:20 on Jul 23 read:*“Present-gate token association —*That line is Kimi’s + Opus’s output from the previous night, and Fable’s first fix is precisely the design it names. In Fable’s favour: it re-derived the mechanism from code and proved it live with its own instrumentation, and only opened the full memory file at 01:04 —**FIXED**— black frames = per-process token table exhausted after 16 pids →`tok=0`\n\nUN-GATED flips;**fix = adapter-global monotonic token**. 2 DEAD ENDS: per-device stamp = bugcheck 0xFC, contiguous retire = permanent wedge.”*after*that live proof, at which point it corrected the note from “FIXED” to “BROKEN AT HEAD”. But it was not searching a blank space.**Kimi inherited a dirty tree; Fable a clean one.** Kimi opened on HEAD plus 736 uncommitted lines of a previous agent’s in-progress work, and spent real effort establishing that the working diff matched the saved`UNSPLIT`\n\npatch. Opus split and committed that work overnight at the user’s request, so Fable opened on a clean tree at the same two commits.**Kimi lost 7% of its span to a client/API incompatibility** unrelated to reasoning quality, and was killed mid-edit by the balance cap.**Fable found something no one had handed it.** The FlipThread vsync-starvation bug appears in no handoff, no memory file, and no prior session. Fable found it in its first 46 minutes by adding its own vsync counters, and its fix (periodic`KTIMER`\n\n+`KeWaitForMultipleObjects`\n\n) survives in the tree today. Kimi had the same counters available and never looked at vsync cadence — that gap is not explained by the head start.\n\nBoth ran at effort `max`\n\nwith a 1 M context window. Kimi\nran through an Anthropic-compatible third-party endpoint with tool search disabled; its context\npeaked around 404 K, Fable’s around 997 K before compaction.\n\n### The day before: Opus 4.8, and what “a lot of hand-holding” cost\n\nThis is the session that produced the ground both later runs stood on — and it is also the least autonomous of the three by a wide margin. 22.9 hours, 533 M tokens, 1,128 responses, 92 separate human or harness interventions, five gate architectures abandoned and three further theories refuted.\n\n#### Composition of interventions\n\n`/goal`\n\nenforcement\nauto-compaction\nThe raw *rate* is nearly identical across all three — 6.6, 6.0 and 6.6 interventions\nper active hour for Opus, Fable and Kimi. What differs completely is the *kind*:\n\n**Opus — 92, of which 55 substantive.** Roughly**19 were corrections of a wrong claim or a wrong path**, on top of 17 interrupts and 16 Stop-hook messages generated by two`/goal`\n\nconditions the user set specifically to stop it giving up and reverting. Median unattended run:**8 responses**. Eleven of the 55 were typed mid-turn and appear only in the queue records.** Fable — 34, of which 26 substantive.**About 9 were human eyes-on-screen reports the model genuinely could not obtain itself, 6 were git hygiene requests, and 6 were design challenges — every one of which produced a real result. One bare nudge in eighteen hours. Median unattended run:**15 responses**; longest, 118 over 63 minutes.** Kimi — 19, of which only 5 substantive:**the opening prompt, a progress-report request, a debugging hint (*“check kdnet, the kmd likely crashed”*), the mid-turn workflow question, and the closing cleanup request.**Ten of the nineteen are bare nudges**— nine`continue`\n\nand one`/compact`\n\n, and six of those nine are the recovery loop for the API-400 fault rather than the model stalling. So: two pieces of technical direction in three hours. Median unattended run:**18 responses**; it ran 156 responses over 104 minutes before the first human word.\n\n### Autonomy, measured three ways\n\n“Can it operate autonomously” resolves into three separable questions: how long does it run unattended, how often does it need *correcting* as opposed to *informing*, and does it self-arrest when it has finished.\n\n- Median unattended runKimi 18\n- Fable 15\n- Opus 8\n\n**156**(104 min) · Fable\n\n**118**(63 min) · Opus\n\n**220**(180 min).\n\n- Corrective inputs / active hourOpus 3.7\n- Fable 1.05\n- Kimi 0.69\n\n*rate*does not.\n\n*does it go down wrong paths or zone in on the solution?*:\n\n**Kimi took the fewest wrong turns per unit of work and needed almost no steering**; it simply converted tokens into verified state slowly, and ran out.\n\n**Opus took the most wrong turns by a large margin** and needed a human to catch three of them — but it was also the only session doing genuinely open-ended search, with no handoff describing the answer.\n\n**Fable is the one that both zoned in and finished**, which is exactly what you’d expect from the run that inherited two days of someone else’s conclusions.\n\n### Verdicts\n\n#### Kimi K3\n\nRight direction · slow conversion · killed early\n\n**Direction: yes.** It identified the correct root-cause class within 105 minutes,\nfound the real defect in the incumbent fix, eliminated one candidate design with a kernel\ncrash dump, and proposed the next one — which was good enough that both Opus and Fable\nadopted it before Fable found the flaw. Six of its ten substantive claims survive\nunchallenged.\n\n**Efficiency: mixed.** Leanest context and cheapest per tool call of the three, but the\nlowest yield per token — two-thirds of its output was internal reasoning, its tool-failure\nrate was 8.0% against Fable’s 4.3%, and roughly a fifth of its wall clock went to environment\nfriction it partly created (CRLF anchors, header hunting, repeated failed greps). When the\nuser pointed that out mid-turn it diagnosed the cause correctly and changed method.\n\n**The real miss** isn’t the superseded design — it’s declaring the multi-second frozen\nframe benign. That was failure mode 2 in the prompt, it had two genuine driver causes, and\nFable found the first one in 46 minutes using instrumentation Kimi had already built.\n\n#### Fable 5\n\nZoned in · finished · inherited a head start\n\nDelivered a complete two-root-cause audit on 20% of Kimi’s budget and a committed, GT1-validated fix on 80%. Then stopped, having done what it was asked. Over the following 15 hours it fixed the scanout path, falsified its own disk-burst theory with a guest ETW trace and said so plainly, ran a controlled experiment on a UMD threading cap and rejected it, reverted a design it had shipped hours earlier once it proved it wrong, and rewrote history to remove it — verifying the final tree bit-identical to the benchmarked one.\n\n**Caveat that matters:** its opening context contained Kimi’s and Opus’s conclusion in\none line, including the fix design and both dead ends. The iso-token comparison is not a\ncold-start comparison. Its independent contribution is the vsync-starvation find, the\nper-thread association, the flip queue, and the UMD pacing — none of which was handed to it.\n\n#### Opus 4.8\n\nHardest problem · most steering · foundational output\n\nThe only session doing open-ended search with no handoff describing the answer, and the\none that paid for it: eight abandoned architectures, three claims retracted only after the\nuser challenged them, 17 interrupts, and two `/goal`\n\nStop-hook conditions written\nspecifically to stop it giving up and reverting.\n\nIt also found the bug that explained why every gate design had failed — the recycled manual-reset proxy fd — and shipped four commits across four repos plus the entire measurement apparatus the other two sessions used. Its 21-minute rescue of Kimi’s broken tree is the sharpest work in the whole dataset: diagnose a bugcheck, fix it, refuse a passing measurement, prove the underlying design unsound, replace it, re-measure, hand off.\n\nThe honest reading is that heavy hand-holding was *working*: every correction\nlanded and moved the investigation forward. It just needed a lot of them.", "url": "https://wpnews.pro/news/bringup-notes-building-triton", "canonical_source": "https://blog.getutm.app/2026/bringup-notes-building-triton/", "published_at": "2026-07-24 00:00:00+00:00", "updated_at": "2026-07-25 04:27:56.226179+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence"], "entities": ["Triton", "DirectX 11", "QEMU", "Neptune", "Direct3D", "WDDM", "MSDN", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/bringup-notes-building-triton", "markdown": "https://wpnews.pro/news/bringup-notes-building-triton.md", "text": "https://wpnews.pro/news/bringup-notes-building-triton.txt", "jsonld": "https://wpnews.pro/news/bringup-notes-building-triton.jsonld"}}