Bringup Notes: Building Triton

We introduced Triton, a DirectX 11 driver for QEMU and you should read that other post about the technical and architectural details of the work. This post is focused on how we did the work. Just like with the Neptune post I want to make an argument for how AI tools cannot replace developers but instead can amplify their work.

To set the stage, I need to explain why developing a DDI driver for virtualization is a hard problem (and why I had to lean on AI tools):

• There is very little documentation and examples online of fully working WDDM drivers and DirectX DDI implementation. The definitive source of information comes from MSDN whose documentation on these topics are genuinely helpful locally (reading structure descriptions for example) but useless globally (understanding end-to-end flows). This means that it is much easier to debug a broken driver than it is to build one from scratch.
• The work spans user/kernel boundaries, guest/host boundaries, and CPU/GPU boundaries. Each boundary on its own come with a set of debug challenges that gets exponentially more difficult when you combine them. Even the fact that the AI tool is running on a macOS host and constantly has to talk to a Windows guest was the source of many wasted turns.
• Success can be hard to quantify. When Anthropic built theirC compiler, they had a large corpus of inputs with a reference implementation to query in order to determinecorrectness. This kind of challenge can be brute forced by an LLM: try everything until the test passes. When building a graphics driver, you either need an existing collection of tests (such as the Vulkan CTS) or some A/B comparison of rendered frames. DirectX has some tests but they are no where near rich enough to build a driver just by observing test results. - Failure can be hard to quantify. How will the AI know that every other frame is being dropped? How will the AI know that some elements are drawn incorrectly? How will the AI know that while everything is rendered correctly and no frames are dropped, the performance is 60% of what the hardware can achieve? These are all real problems that had to be solved.

All of these challenges are difficult for humans and they cannot be one-shotted by an AI assistant. Where the AI can help is to do the “grunt work” of trying out different designs, testing hypothesis, and building tools and harnesses for other agents. AI works the best autonomously when either the problem space is narrow or the success criteria is wide. AI tends to fall on its face when you have a large problem space with a narrow success criteria. When I hear about “impressive” work done by AI assistant without a skilled operator, it always falls into one or more of the following groups:

• It is a visually impressive demo but not actually technically challenging. (Wide success criteria)

• It heavily relies on existing work. (Narrow problem space)

• It is not useful or working beyond the demo. (Both) So what is the value of the human operator? Their job is to break a large and intractable problem into a series of smaller ones that is friendly for an AI to solve… and more importantly for AI to validate. Don’t waste your time trying to prompt engineer or write skills or benchmark what the best coding harness is. All of that is table setting. The one skill you need to master is the same skill you needed before AI tools existed: the ability to break apart complex problems.

Forty days, on the record #

The Triton bring-up started in late May, after the completion of the Neptune bring-up. There was a large break when Fable became unavailable because I had (correctly) hoped that it would be back soon (just not as soon as I had hoped).

The timeline

Each marker is coloured by the model that did the work; a split marker means the milestone spanned more than one model. Counts on the chips are user turns.

The full bring-up, milestone by milestone33 entries · May 20 – Jul 23 #

• May 20

VirtualBox teardown → Triton is designed

A study of how VirtualBox runs D3D11 in a VM (DDI → SVGA3D command stream → hypervisor → host D3D11) ends with the decision to collapse the middle two layers:

Triton becomes a user-mode library that turns Windows D3D11 DDI calls straight back into D3D11 API calls, which Neptune already forwards to the host.Opus 4.718 - May 20

DDI surface and the DXBC container

The DDI entry points get built out and inventoried. A first hallucinated

TritonDxbc

is thrown away and rewritten directly against VirtualBox’sDevVGA-SVGA3d-dx-shader.cpp

— known-working code — with the chunking and command-encoding layers stripped out.Opus 4.732 - May 22–23

Windows KMD/UMD groundwork

In parallel on the Windows and desktop machines: viogpu3d build automation, an ARM64 build port, patch triage across three diverged driver forks, and a variant analysis of the async-path fixes.

Opus 4.7188 - May 24

Every DDI cross-checked against MSDN

A full review pass: each DDI function looked up on MSDN, compared against both the docs and the VirtualBox implementation, written up in

DDI_REVIEW.md

, then every finding independently re-validated and fixed. Followed by a licensing, naming and comment cleanup.Opus 4.720 - May 25

Neptune wired into the Windows build

virtio-win-mesa (UMD) + viogpu3d (KMD) + neptune-protocol all building on the Windows dev VM, and a two-VM development loop stood up on the Linux box: dev VM builds, test VM runs under

-snapshot

, WinDbg over COM2, artifacts shared through virtiofsZ:\

.Opus 4.741 - May 26–28

Present path lands — and a TDR loop blocks the desktop

The DXGI present proposal is implemented on both sides and verified (

FlushToScreen scanout blob valid=1 1920x1080

, 54 clean swapchains). The desktop still doesn’t come up: a TDR → bugcheck → reboot loop is root-caused to an unregistered shader-resource view tearing down the context — a pre-existing DXVK shared-resource bug.Opus 4.741 Opus 4.86 - May 28 – Jul 1

Guest-side shared handles

Neither DXVK nor the future backends can pass shared handles through, so they get emulated on the guest using the fence and context-blob machinery already in place. The session runs across the four-week gap; it ends by proving the live desktop capture works and cleanly

disproving app-liveness via draw-RT capture — DWM layers app windows through DirectComposition, invisible to the render server.Opus 4.871 - 6 days quiet

• Jun 4–5

DXGI DDI study

The two virtualization drivers’ DXGI swapchain DDIs put side by side — Neptune’s WIP implementation against VirtualBox’s — to settle the architectural questions before the rework.

Opus 4.717 Opus 4.83 - 5 days quiet

• Jun 10

d3dmetal-native begun

libd3dmetal-native

starts on the laptop: a native macOS library implementing the GFXT interface on top of Apple’s D3DMetal.framework — essentially dxvk-native but backed by Metal. It becomes the macOS host renderer that Triton talks to two weeks later.Fable 55 - 20 days quiet

• Jul 1

Clean-slate re-bring-up

Everything the previous agent had stashed stays stashed. A fresh context re-reads Neptune, Triton, KMD and UMD from scratch, reviews before writing, and rebuilds the whole host and guest stack. Part of that pass was confirming that

no VirtualBox code remains in the Triton driver— VirtualBox stays a behavioural reference only, consulted for what a correct DDI implementation has to do (the “VBox-parity” DDI surface, and the find thatSetDisplayMode

was a silent stub), never a source of code.Fable 58 - Jul 2 ★The Windows desktop rendersLive desktop up in the QEMU window— user Active, presents flowing, zero errors. The last two crashes go with it: a KMD blob-info race (fixed with a locked snapshot + degenerate-scanout deferral) and a QEMUqemu_memfd_alloc

abort, now structurally impossible via scanout-blob rect validation.Fable 58 - Jul 2–3

dmabuf replaces the fake shared handles

The emulated handles come out. DXVK gets real dmabuf import/export, and virglrenderer and guest Mesa drop the single-host-device / threaded-context sharing hack.

MixedFable 518 Opus 4.87 - Jul 3

Artifact triage: glyphs, icons, compositing

With the desktop up, the rendering is wrong in interesting ways — wrong glyphs, half-noise icons, fully transparent windows with working title bars. The first job is a way to diagnose it without a human looking at screenshots.

Fable 523 Opus 4.84 - Jul 4 ★Fire Strike runs to completionA full 3DMark Fire Strike run finishes

EXIT 0

with real scores (Graphics 1347 / Physics 1858 / Combined 369), rendering live to the GTK window — Demo and GT1 playing fullscreen with FlushToScreen past 900 presents.Opus 4.810 - Jul 5

Neptune on macOS: the cube spins

The other front: Neptune ported to macOS over d3dmetal-native, running under Wine + FEX in QEMU. First green run — 3,603 presents, zero crashes — after gating the Venus ring watchdog on CPUID emulator detection.

Opus 4.826 - Jul 6

The perf ceiling is not the driver

Fire Strike GT1 is pinned at ~8 FPS and every driver-side lever moves it by nothing — which is the proof. GPU 99% busy at 300 MHz, package power 4 W of 50 W: a

30-amdgpu-pm.rules udev clamp forcingpower_dpm_force_performance_level=low

on every boot. The >5000 goal was unreachable, with receipts.Opus 4.828 Fable 55 - Jul 6

Stability: ten clean cycles

Ten consecutive boot → install driver → Fire Strike → screenshot → calc.exe → screenshot cycles, plus the fix for a long-standing wedge where the GTK window froze right after driver install.

MixedOpus 4.89 Fable 54 - Jul 6–8

WSI moves to the guest; repos cleaned for upstream

Host-side swapchain creation is removed entirely — the guest driver owns swapchain management and shares textures for scanout. virglrenderer, Mesa and DXVK histories are rewritten, diagnostics stripped, and the whole stack re-tested end to end.

Opus 4.891 Fable 55 - Jul 9

Windows-on-ARM bring-up starts on Apple silicon

The desktop machine takes over: Windows on ARM64 under QEMU on macOS, Triton talking to d3dmetal-native on the host, KDNet debugging into a Windows dev machine over SSH. Nothing renders yet.

MixedOpus 4.816 Fable 511 - Jul 10 ★Desktop, taskbar and GDI apps render on ARM64Five root fixes land: the DXBC converter SIGSEGV (ISGN scalar-mask widening), a KMD paging bugcheck, a GPU-residency wedge, vrend↔neptune shared-memory aliasing (which unblocked all window content), and host-side A8→RGBA8 translation for DWM’s glyph atlas. regedit is pixel-perfect; GPU-drawn glyphs are still wrong.

MixedOpus 4.843 Fable 516 - Jul 12

Fire Strike perf pass on Apple silicon

Fire Strike made to launch, then profiled on both sides of the boundary — guest KMD/UMD and host render server — with symbols resolved, until the workload is GPU-bound.

Opus 4.831 - Jul 13

A blank-frame detector in the host

Every few frames come back solid black or a flat color. Rather than eyeball screenshots, the host maps the first page of each frame and samples ~1024 pixels, so the bug can be reproduced and counted automatically.

Opus 4.877 - Jul 14–16

The colour/RG-swap hunt

Fire Strike’s demo tints red or green and GT1’s particle effects come out the wrong colour — green flames. The hunt runs through Triton’s workaround code and D3DMetal-native.

Opus 4.838 Fable 58 - Jul 15

DXBC container review finds the miscompile class

A review of Triton’s DXBC container implementation against VirtualBox’s translator and the other reference implementations, aimed at the subtle shaper bug that makes the host translator miscompile some shaders.

Fable 55 - Jul 16

DXMT added as a second macOS backend

dxmt-native gets fence and texture export/import over shmem and is wired into virglrenderer alongside d3dmetal-native — and unlike D3DMetal it builds native ARM64, no Rosetta. Linux + Wine validates first, then Triton + Windows.

MixedOpus 4.815 Fable 511 - Jul 17 ★Fire Strike to completion on refactored DXMTThe refactored DXMT renders Fire Strike to completion at

4989(Graphics 6213 / Physics 5068 / Combined 1995) on the fast host — confirmed against the exact HEAD build, with the earlier “renders nothing” shown to be intermittent rather than a break in the refactor.MixedOpus 4.814 Fable 59 - Jul 17

arm64x packaging

The Triton UMD is packaged as arm64x — arm64ec and x64 in one binary — through a dedicated build step rather than out-of-band scripts, the KMD gains a user service, and a full build-and-test cycle runs against both host backends.

Opus 4.872 - Jul 18

DWM layer drops traced to the impostor texture

Windowed apps lose whole composition layers on D3DMetal-native — empty start menus, missing search boxes, borderless windows. The trail leads into

substitute() , where shmem-backed impostor textures stand in for shared textures so they can cross process boundaries.Opus 4.842 - Jul 19

Device-lost and black-screen hangs fixed

Two black-screen variants — immediately after driver install, and after a complete Fire Strike session — chased through KDNet, guest and host tooling; plus an audit of the impostor-texture mechanism against Apple’s docs.

Opus 4.899 - Jul 20

History collapsed to one Triton commit

Upstream prep: every follow-up commit that belongs to Triton is rolled into a single introducing commit, with Neptune-side and unrelated changes kept separate. Performance characterisation of the D3DMetal-native path begins.

Opus 4.857 - Jul 21–22

Present-gate pipelining

Present was blocking on GPU work, leaving both CPU and GPU under-utilised. Unblocking it let presents race ahead into black frames; the fix moves the gate out of the UMD DDI and onto the KMD mechanism designed for it.

Opus 4.870 Kimi K316 Sonnet 57 - Jul 23 ★Final numbersBack-to-back clean Fire Strike runs on the Windows guest:

5124 on DXMT (ARM64) and5682 on D3DMetal (x86_64/Rosetta), all four workloads,EXITCODE=0 , black-frame rate ≤0.8%. The Linux/Wine reference on the same host scores7624.Fable 5** 54Opus 4.8 6** - Jul 23

Upstream review pass over UMD and KMD

Interactive, item-by-item review of the guest driver code — development narratives out of the comments, bring-up log entries accounted for, MSDN contracts re-checked — ahead of publication.

Fable 516

User turns per day

1418 Opus 4.7

424 Fable 5

275 Kimi K3

16 Sonnet 5

7 No reply

38

Table view — turns per day by model #

Day	Opus 4.8	Opus 4.7	Fable 5	Kimi K3	Sonnet 5	No reply	Turns	Est. cost
2026-05-20	62	62	$225.98
2026-05-21	7	5	12	$2.18
2026-05-22	167	167	$330.36
2026-05-23	43	43	$204.83
2026-05-24	26	26	$66.77
2026-05-25	37	37	$224.60
2026-05-26	23	23	$404.88
2026-05-27	31	1	32	$298.21
2026-05-28	12	6	18	$266.97
2026-05-29	30	30	$573.48
2026-05-30	8	1	9	$53.71
2026-06-04	3	7	10	$4.96
2026-06-05	10	10	$4.70
2026-06-10	5	1	6	$85.90
2026-06-28	14	14	$272.97
2026-06-29	11	11	$351.75
2026-06-30	8	8	$192.34
2026-07-01	15	5	13	33	$486.88
2026-07-02	24	28	52	$553.05
2026-07-03	10	55	65	$448.39
2026-07-04	25	25	50	$876.18
2026-07-05	58	1	59	$471.15
2026-07-06	133	13	146	$599.92
2026-07-07	55	1	56	$407.79
2026-07-08	59	3	62	$107.37
2026-07-09	53	1	29	3	86	$664.24
2026-07-10	3	11	14	$597.21
2026-07-11	11	3	14	$524.09
2026-07-12	97	2	99	$483.60
2026-07-13	62	62	$421.32
2026-07-14	46	3	49	$377.41
2026-07-15	31	2	33	$261.41
2026-07-16	5	22	27	$676.39
2026-07-17	153	6	159	$438.89
2026-07-18	133	5	138	$524.43
2026-07-19	137	2	139	$513.46
2026-07-20	97	2	99	$542.57
2026-07-21	92	92	$568.66
2026-07-22	10	16	7	33	$113.46
2026-07-23	23	70	93	$537.50
Total	1418	424	275	16	7	38	2178	$14,760

Two things fall out of that chart. The bring-up is not a smooth ramp — it is four bursts separated by two multi-week gaps, and the busiest days are all in the last fortnight, after the driver already worked. And the 2,178 prompts are spread over just 40 active days, which is roughly 54 typed messages on a day when anything was happening at all.

Daily token usage

Model usage

• Opus 4.848.2M67.3%
• Opus 4.711.1M15.5%
• Fable 511M15.4%
• Haiku 4.5353K0.5%
• Kimi K3926.7K1.3%
• Sonnet 52.66K<0.1%

Table view — per model #

Model	User turns	API turns	Input	Output	Cache read	Cache write	Est. cost
Opus 4.8	1418	31,683	2.02M	46.2M	11.5B	152.7M	$7,887
Opus 4.7	424	8,054	307.3K	10.8M	2.63B	40.9M	$1,840
Fable 5	275	10,531	1.07M	9.94M	4.05B	37.3M	$5,018
Haiku 4.5	0	1,383	18.9K	334.1K	64.5M	2.59M	$11.37
Kimi K3	16	274	763.4K	163.3K	70.5M	0	n/a
Sonnet 5	7	12	24	2.64K	5M	425.7K	$3.14

Where does the human fit in? #

My subjective feelings were that I had just as much cognitive load–if not more–than writing code without any AI. The shape of that load was different though. When writing code, the difficulty stems from trying to keep track of all the moving pieces in my head and trying to convert these abstract ideas into concrete code. The most rewarding feeling is seeing the code work after hours or days of coding. For this project, I have not written a single line of code myself. However it is still the same feeling of having these abstract ideas and trying to guide the AI into producing something that feels correct. A lot of the work is in reading the code and in debugging issues (just like before). Once a part is complete and I read the code and it looks good and I run it and it works, I get the same feeling of satisfaction.

The shape of it · One person, 59,443 words, 45,893 tool calls

Over nine weeks of bringing up a Direct3D 11 driver stack across QEMU, virglrenderer, Mesa, a Windows WDDM kernel driver and a Metal backend, the human typed 1,576 messages totalling 59,443 words — about 120 paperback pages. The models answered with 101,031 messages, 45,893 tool calls, 67.4M output tokens and roughly 1.84 million words of prose.

That is a 31:1 ratio of model words to human words. Put differently: every word the human typed bought about 0.77 tool calls, and every hour of model work cost about 125 human words.

This is the clearest evidence for what I said at the beginning: the tools allow you to amplify your existing skills. In this case, ~60 hours of human work amounted to ~480 hours of machine work.

Finding 1 · It is not a continue button

The cheapest story about AI-assisted engineering is that the human degenerates into a keystroke: continue, go ahead, yes, ship it. The transcripts say otherwise, and they say it precisely — because “continue” is a short, exact string that can be counted.

Of 1,576 human messages, 138 carried no new information: 88 pure continuations (continue

, yes

, go ahead

, keep going

), 36 status polls (ping , any updates?

, is it done?

) and 14 arbitrations where the human picked among options the model had laid out (try option 1

, let's do B1+B2

). That is 8.8%. The remaining 91.2% carried content, and they carried 99.5% of the words.

There is a twist worth noticing. When the human did just say “continue,” it bought more autonomous work than an average substantive message — a median of 15 tool calls against 7. “Continue” is not a filler turn; it is the specific instruction keep grinding on the thing you already understand, and it is issued precisely when the model already has enough context to grind.

Finding 2 · Zero messages about how to use a tool

Across 1,576 human messages there is not one instruction about search or file mechanics. No “grep for”, no “use ripgrep”, no “cat that file”, no “open the header”. Zero.

Meanwhile the models made 29,081 Bash calls, 6,293 edits, 5,968 reads and 1,048 writes. The entire mechanical layer of the work — deciding what to search, which VM to SSH into, which build script to run, where the log went — was never once specified.

What the human does specify is one level up, and it is remarkably consistent:

The human’s actual vocabulary	Messages	Share
Cites a specific git commit SHA (207 distinct SHAs)	135	8.6%
Names a specific source file	129	8.2%
Invokes MSDN / a spec / an API contract as ground truth	66	4.2%
States a standing rule the model should carry forward	97	6.2%
Tells the model how to search, read, or grep	0	0%

This is what “focusing on the big picture” looks like when you go measure it. The human’s register is commits, DDI entry points, render encoders, MSDN contracts and architectural trade-offs. The register of tool calls, file paths and shell invocations simply never appears — not because the human suppressed it, but because it never came up.

“One level up” is a good way to put it. I have been thinking about how in the early days, people programmed in machine code. Then assembly code was invented to abstract away the encoding of instructions so human can focus on the semantics. Then higher level languages like C and Python were invented so humans can focus on the logical structure rather than the semantics. Now, with AI assistants, we can focus on the design and architecture rather than the logical structures.

Finding 3 · The model explains; the human decides

36.5% of substantive messages are questions or requests for explanation — 525 of 1,438. And these turns behave completely differently from the rest of the corpus.

Questions are cheap for the model and expensive in value for the human. They cost a couple of minutes, they mostly don’t touch code, and 56% of the time the human’s very next message is a directive — the explanation gets converted straight into an instruction. The pattern is visible turn by turn: ask → act

is 15.1% of all turn-to-turn transitions, almost exactly balanced against act → ask

at 15.7%.

The mechanism is worth naming, because it is the actual source of the human’s leverage. The human does not need to have read the 400,000 lines of VirtualBox, dxgkrnl, DXVK and D3DMetal in this project. They need a model that can read them on demand and explain the call chain — and then their own domain judgement is enough to say that’s wrong, DXGI should already be limiting inflight frames, check the MSDN contract. Explanation is what converts one person’s judgement into coverage over a codebase they could not otherwise hold in their head.

I have found that AI models are much, much, better at understanding code than writing new code. Even if you are against using AI tools to write code (and that is an understandable position given how much work it is to properly review the code), you should be using it to understand new codebases.

Finding 4 · Reading the generated code is a job, and it has a measurable footprint

Fourteen sessions open with an explicit protocol the human wrote, some version of: “We will be doing an interactive code review. Each review item is either a question or a request for change. If it is a question, do not jump to changing the code.”

Those 14 sessions contain 183 human messages against 2,626 tool calls — 14.3 tool calls per human turn. Everywhere else in the corpus the figure is 31.1. Review mode is more than twice as human-dense as the rest of the work, and it eats about 25 engaged hours.

The extreme case is a rebase review on 2026‑05‑22: 92 human messages, 566 tool calls — 6.2 tool calls per turn. That is a person going through a commit history line by line with a machine doing the typing.

What review turns up is not stylistic. A sample from one three-hour session on virglrenderer:

npt_context_import_resource

we removed the code for `close(fd)`

then set `res->u.fd = fd`

. When we have VIRGL_RESOURCE_FD_SHM

does this not leak res->u.data in the union? this change was added in context of d3dmetal-native but is this change safe for all backends including dmabuf on linux?”npt_cs.h

that is just an empty line. drop that change. also remember for the final pass to do the same in other commits as well.”And the corrections land. In 1,576 messages there are only 24 cases of the human having to repeat themselves to consecutive turns — 1.5%. Pointing at the problem is usually sufficient; the model does not generally need to be told twice what the fix is.

What it does need is emphasis. 30.1% of human messages contain shouted words — ALWAYS, NOT, ONLY, NEVER, DO NOT — and the targets are consistent: don’t guess, don’t hand-wave, don’t push, don’t commit diagnostics, don’t stop.

Finding 5 · The expensive problem is that the model cannot see

This project’s hardest bugs were visual: a black screen, a missing search pill in the Windows taskbar, glyphs that don’t render, a magenta tint over an aurora, frames that flash black at 30 Hz. The model has no eyes. For nine weeks, the human was the display.

139 messages are the human reporting what is on a screen the model cannot query. Read in date order they read like an instrument log, and the language sharpens over time:

May 28— “I do see the desktop after the driver load but it seems to have loaded one frame and the entire process is frozen”

Jul 4— “I see the calculator now but as I move the mouse cursor around, there are random artifacts that appear… once the mouse stops, after a few seconds, it refreshes and looks right again”

Jul 17— “I no longer notice the hangs but now we have a constant micro-stuttering. As GT1 runs, I constantly see flashes of black screen that feels like 1-2 frames of black. It is persistent through the entire run.”

Jul 23— “I see a completely black screen for 17 seconds. Then I see frame 801 stuck for the next few seconds. Then it terminates.”

That progression — from “frozen” to “1–2 frames of black” to “17 seconds, then frame 801” — is the work. The human is converting a perception into a quantity the model can go look for. Until that conversion happens, the model is guessing.

The transcripts contain the moment where this becomes explicit, and the human is blunt about it twice:

The measurable version

A message reporting a visual symptom with no instruction about how to observe it buys a median of 8 tool calls. A message that tells the model what to instrument, dump, trace or capture buys 18 — and runs 11.2 minutes instead of 7.4. Same human, same bugs, same models. The difference is whether the model has been handed a way to see.

I do feel this is a problem that will be solved with better models.

Case study: the missing search pill

On 2026‑07‑18 the human opened a session with a description of a rendering bug that had persisted “since day 1”: under the D3DMetal backend, the Windows desktop drops compositing layers. The Start menu is sometimes an empty box; the taskbar’s rounded search box renders as bare text on the bar. Twenty-seven human messages and 705 tool calls later, over fourteen hours, the shape of the human’s day is this:

The arc is the whole thesis in one day. It opens with the symptom described in words. The model immediately starts bisecting compiler workaround flags, and the human has to stop it — three times, twice by interrupt:

Then eighteen turns of hypothesis and small experiments — the 2DTextureArray view, CheckFormatSupport

, whether DWM ever queries D3D11_FEATURE_FORMAT_SUPPORT2

— mostly cheap runs of 2 to 40 tool calls. At turn 18 the human asks for the first real data channel: “can you dump the DXBC → Metal shaders for each DXBC. try to find the one that composites the search pill. then compare the metal code for both.” At turn 19 they escalate it to “get the AIR of both sides and compare them” — and the model runs 191 tool calls, then the human disappears for six and a half hours.

And then the ending, which is the part worth sitting with. The human comes back having done the observation work themselves:

dwm-render-debug . I’ve dumped the PNG for each render encoder color 0 attachment output at the end of each render encoder. The last image shows the rendered taskbar. In DXMT, the taskbar is complete: clock, icons, search pill. In D3DM, it is missing many of those elements.”Followed by pasting a raw Metal command stack into the chat — “do you see the missing draws” — and then, two turns later, explaining the file format to the model: “each empty line separates a render encoder.”

The bottleneck was never the model’s reasoning. It was the absence of a channel between a pixel on a screen and a token the model could read. Building that channel took a human most of a day.

There is a second-order effect worth flagging: 48% of a session’s human messages fall in the first third of its span. The human’s work is front-loaded, and the session-opening brief is the highest-leverage thing they write — a median 52 words against 18 for later turns, buying 17 median tool calls against 8. Roughly half the messages in a session are spent aiming it.

Also: I cannot sit here and watch the whole thing.”

Once I was able to get the AI to empirically measure the failure and set a goal to fix that metric, the hard work is “done” and the rest can be automated.

Finding 6 · The human compiled their own continue button

Because they could not sit there and watch, they wrote down what “done” means and made the harness enforce it. Across the corpus there are 49 distinct /goal conditions — success predicates registered as stop-hooks — that fired

395 times. Of those,

367 fires blocked the model from ending its turn.

Three hundred and sixty-seven times, the model tried to stop and a sentence the human had written earlier pushed it back in. That is the “continue” key, turned into a spec.

Goal condition (abridged)	Fires	Blocked
“Figure out the difference between DXMT and D3DMetal in pill rendering… Keep going until you figure out why the pill is not rendered, confirm it, patch in a fix, and test that it works.”	56	56
“Fix all the P1 items. Make sure to follow the same pattern: validate API calls with MSDN documentation, cross-check against virtualbox, and commit without narrative comments”	48	48
“Fix all three issues… Make sure the VM can run for 15 minutes without crashing or going out of memory while maintaining visible scanouts.”	35	35
“fix the black screen”	35	34
“Achieve > 5000 on 3dmark firestrike”	23	23

What is striking is not the goals that name an outcome. It is how many of them name a method. The human is not only specifying what done looks like — they are specifying the epistemics:

And separately, typed into the conversation as a standing instruction:

do not prompt me for guidance unless you cannot follow these rules.”

That last clause is the human explicitly pricing their own attention.

Finding 7 · Less hand-holding, quantified

I felt that Fable 5 required a lot less babysitting than Opus 4.8. Here is the data to back it up.

The corpus spans three model generations on overlapping work, which makes “this one needed less babysitting” a testable claim rather than an impression. It holds.

Model	Runs	Median tools	Mean	p90	Median run	Interrupted
Fable 5	222	17	53.8	152	9.1 min	14.4%
Opus 4.8	1,109	8	24.0	63	5.1 min	22.3%
Opus 4.7	253	6	27.5	92	2.5 min	12.6%

Model choice was not randomised, so the honest check is within-project, where the task mix is at least comparable:

| Repository | Fable 5 median | Opus 4.8 median |

|---|---|---|
| Neptune-QEMU | 21 (n=94) | 9 (n=445) |
| VMs (driver bring-up) | 16 (n=48) | 11 (n=220) |
| kvm-guest-drivers-windows | 19 (n=7) | 7 (n=50) |
| dxmt | 38 (n=4) | 5 (n=17) |
| d3dmetal-native | 6 (n=49) | 14 (n=31) |

Four of five go the same way, one goes the other. The two longest unattended runs in the entire corpus are both Fable: 554 tool calls over 6 hours and 541 tool calls over 13.5 hours, each launched by a single human message.

The “but I still have to step in” half is equally visible: one Fable run in seven was interrupted by the human hitting escape. Across all models it is 19.0% — 321 runs, 334 interrupt events. Roughly one intervention every five turns, all corpus long, at every model generation.

Finding 8 · The human got more involved over time, not less

This is the result that runs against the intuition, and it is not an artefact of session count — it survives normalising by engaged hours.

Over the same period the mix of what the human was saying shifted decisively. The share of messages carrying a data-collection instruction went from 9% to 24%. Tool calls bought per human message fell from 39 to 24.

The explanation is in the nature of the work, not the quality of the models. Late May was greenfield: make anything appear on the screen. That is a goal you can state in one sentence and walk away from — and the human did, at 1.86 messages per hour. Late July was make the taskbar composite correctly, eliminate the 40% black-frame rate, get Present off the critical path, and clean the commit history for upstream. Those are goals where the human is the oracle for whether it worked, the reviewer of every line that ships, and the designer of the measurement.

The pattern

Autonomy is highest when the goal is coarse and failure is obvious. It drops exactly where the work gets valuable — where “correct” is a judgement call, where the evidence is a pixel, and where the code has to be good enough for someone else to merge.

This makes sense because the initial bring-up phase was mostly just guiding the AI into a good design. The debug (correctness and performance) phase required a lot more intervention due to the previously stated difficulties.

So, where? · The human is the part that cannot be delegated because it touches the world

Reading nine weeks of this backwards, the human’s residual job sorts into four things, and none of them is writing code:

Perception. The model cannot see the screen, hear the fan, or feel that the cursor lags. 139 messages exist purely to carry a perception across that gap, and the hardest single day in the corpus was spent building a pipe so the model could see one taskbar.Judgement about what “correct” means. 49 goal predicates, 97 standing rules, 66 appeals to a spec, and an interactive review protocol the human wrote and ran fourteen times. The model can check a program against a rule; someone has to author the rule.Decomposition and altitude. The session-opening brief is the single highest-leverage artefact in the corpus — 3× longer than an average message and buying 2× the autonomous work. Zero of 1,576 messages descend to tool mechanics.Being the world’s actuator. Rebooting the host, restarting the VM, and telling the model what happened.

The volumes are worth restating plainly. The human spent about 65 engaged hours on the loop across nine weeks and produced 59,443 words. The models spent 476 hours and produced 1.84 million words plus 45,893 tool calls. The leverage is roughly 7:1 in time and 31:1 in output.

But the leverage is not free and it is not increasing. Over the nine weeks the human’s rate of input doubled — because the work moved from “make it render” to “make it render correctly,” and correctness is exactly the region where the model needs a person to be the eyes, the spec, and the reviewer. The bottleneck did not disappear. It migrated, from typing to seeing.

Appendix: one bug, three models #

One entry in that timeline supports a comparison the rest of the project does not. Present-gate pipelining — the second-to-last milestone, 21–23 July — was worked by three different models on three consecutive days, two of them from a verbatim-identical opening prompt. The next part compares them at equal token spend. It is not a benchmark, and the fairness ledger below matters more than the headline; but it is a rare look at what “needs less hand-holding” means when it is counted rather than felt.

When Kimi K3 came out, many people online claimed it was as good as Fable 5 so I was curious to test the claim out. This analysis is not a fair comparison of capability because I limited my Moonshot token spend to $30 but I think what the model did with that $30 is an interesting story to read.

The three sessions

All three ran on the same machine against, driving the same Windows-on-aarch64 guest through the same nept.py

harness.

Opus 4.8	Kimi K3	Fable 5
Session	a0816380	d8f976e3
ff50fe7a → daff6027
Model phase	Jul 21 11:05 → Jul 22 09:58	Jul 22 11:58 → 15:11	Jul 23 00:20 → 18:14
Wall clock	22.9 h	3.21 h	17.9 h
Active time (gaps ≤5 min)	13.9 h	2.90 h	5.71 h
API responses	1,128	274	611
Tool calls	1,101	301	599
Output tokens	1,798,997	163,274	612,249
Billed tokens	533.1 M	71.4 M	322.0 M
Spend	$339 list	$30 hard cap
$392 list
Ended because	user asked for cleanup + handoff	429 · insufficient balance
user exited after a regression check

71.4 M billed tokens— 763 K uncached input, 163 K output, 70.5 M cache reads, zero cache writes. That number is the yardstick for everything below.

The problem all three inherited

A paravirtual WDDM driver stack: guest UMD (`virtio-win-mesa`

, the Triton D3D11 driver) → KMD (`kvm-guest-drivers-windows`

) → virtio → host virglrenderer

→ QEMU cocoa → Metal. Present was blocking the app thread while the GPU worked, so CPU and GPU never overlapped: 3DMark Fire Strike GT1 ran at ~27 fps with the render thread parked inside Present

.

Removing the block unlocked pipelining and broke correctness. The user’s opening prompt — identical, verbatim, to Kimi and to Fable — named three failure modes and told the model to distrust every prior conclusion:

“…audit the code at HEAD (KMD+UMD) focusing on the events, fencing, and present architecture… Then build and test the driver and try to detect the different failure modes we’ve observed: 1) flashing black frames (likely inert fences → present before frame is ready), 2) stuck frames (likely stuck fences → present gets blocked in the pipe), and 3) underutilized GPU (due to lack of pipelining). The existing test and diagnostics the last agent produced is incomplete.”

Opus 4.8’s prompt the previous day was different in form — it began

from HANDOFF-perf-analysis-methodology.md and asked for triage and a diagnostic A/B path rather than an audit of a specific HEAD.

Was Kimi K3 going in the right direction?

Mostly yes. Its progress report at 13:43 made ten checkable claims. Six hold up against everything measured afterwards, two are correct for the exact tree it audited, and two are wrong — one of them badly, because it closed off the failure mode the user had explicitly asked about.

Kimi’s claim	Verdict	Adjudicated by
Failure mode 3 (GPU underutilisation) is already fixed at HEAD — overlap index ≈1.1, GT1 50.2 fps DXMT / 77.6 D3DMetal vs the 27.2 fps blocking baseline
Right	Fable measured overlap ≈1.0 on the same HEAD independently the next morning
The broken piece is the association between a fence arm and the flip it gates — flips run un-gated (tok=0 ) → black/torn frames
Right	Fable’s independent audit reached the same conclusion and proved it live: NPT-DIAG promote tok=0 done=145 zero=129 nonzero=0 . The fix that finally shipped, 5d984dc3 , is a token-association fix
The mechanism is the 16-slot pid-keyed token table exhausting after 16 presenter pids, then dropping silently
Right for its tree	True of HEAD + the 736 uncommitted lines Kimi audited. At clean HEAD the mechanism is different — the escape lands on the npt transport’s own private D3DKMT device, so every flip reads 0 from the first frame. Both are correct; they audited different trees
A per-device stamp via a user-mode : D3DKMT handle is impossibleVioGpuDevice::FromHandle calls a virtual Magic() on a foreign pointer, and DXGK_HANDLE_TYPE only has ALLOCATION and RESOURCE
Right	Kimi built it, bugchecked the guest with 0xFC, and diagnosed why. Recorded as do-not-retry and never revisited by anyone. This is its most durable contribution
The fix is one adapter-global monotonic token, peeked by the flip at latch time
Adopted, then killed	Fable shipped precisely this as b2b04618 , then proved it livelocks a pipelined app — at depth 3 with steady frame times flip N+1 always latches before token N retires, so each new latch replaces the almost-ready one. Reverted, then dropped from history
Its own follow-on redesign: contiguous-prefix retirement, so done ≥ T implies every token ≤ T retired
Unsound	Completions aren’t 1:1 with arms in token order (PFENCE-ARM #5632 token=5632 vs PFENCE-CB #5632 token=5730 , wrap=72 ), so the walk wedges permanently — done frozen at 651 while flips asked for 2822. Caught by Opus 4.8 1h58m after Kimi’s budget died; Kimi never compiled it
Stuck frames are not stuck fences — the 2–4 s mid-GT1 gaps are app-side guest CPU
Right	Fable’s 51-second guest ETW trace: the frame-producing thread is 92–100% on-CPU straight through the hitch window, waits ≤8%, disk utilisation ≤30% — single-threaded asset ingest inside the statically-linked exe
The 30–60 s freezes are an idle desktop with no presents — benign
Wrong	The multi-second frozen frame was real and had two driver causes, both fixed later: FlipThread vsync starvation (every fence-completion wake restarted the vsync period, and MMIO-flip completion is reported only by the CRTC_VSYNC interrupt — a 1123 ms present stall was measured), and unbounded CPU run-ahead (every token in the flip queue seconds from retiring → 5–15 scanouts/s). Fable found the vsync bug in its first 46 minutes
The VIOGPU_GET_GATE_STATS escape the old probe expects no longer exists in the KMD
Right	Fable later re-implemented it as a diagnostic patch
Two stale claims corrected: the KMD comment “the current UMD passes NULL” is false; c88154be 's “1 s bounded fallback” is 10 s in code
Right	Both carried into the handoff document unchallenged
Ten claims	6 ✓ · 2 ◐ · 2 ✗

Was Kimi efficient with those 71.4 M tokens?

Structurally, yes — it was the leanest of the three per unit of work attempted. Behaviourally, no — a large share of the budget went into recovering from its own malformed tool calls and into thinking rather than acting.

Tool-call failure rate

Every failed tool call costs a full context replay to recover from. Kimi’s rate was roughly double Fable’s and an order of magnitude above Opus’s at the same point in their sessions. “Soft failures” include exit code 1, String to replace not found

, Build FAILED

and empty-result greps.

• Kimi — hard errors8.0%
• Kimi — incl. soft12.0%
• Fable — hard errors4.3%
• Fable — incl. soft6.9%
• Opus — hard errors0.9%
• Opus — incl. soft10.8%

Bash 10.3%/

Edit 13.6%; Fable

Bash 5.6%/

Edit 0%; Opus

Bash 0.5%/

Edit 5.0%. Opus’s comparatively high soft rate is mostly deliberate probing — greps and

findstr

queries that legitimately return nothing.#### Where the 3h13m actually went

Inside those blocks, four concrete friction episodes ate roughly 40 minutes — about a fifth of the whole session — and the user eventually asked about it directly:

CRLF versus Kimi tried to editEdit

anchors — ~16 min.viogpum.h

from a reconstructed anchor string; the file is CRLF, so the exact match failed. It wrote an anchor-asserting Python script, that anchor didn’t match either, and it ended up inspecting the bytes withfile ,cat -A

andrepr() . Its own thinking later names the cause exactly:*“my anchors keep being wrong because I’m reconstructing file contents from memory/diffs.”*Fetching one WDK header — 10 turns, ~8 min. Looking ford3dkmddi.h

on the Windows dev box: wrong include subdirectory (km/

instead ofshared/

), then five different shell quotings of the samescp

, before giving up andcopy

-ing it to the home directory first.Eleven consecutive greps chasingSetEventOnCompletion

through virglrenderer, most returning nothing, each one a full context replay.Seven turns to launch a sampler— discovering thatstart /min

doesn’t detach over SSH, then thatnept.py test

blocks.The user eventually asked about it directly — and Kimi took the point. At 15:03, mid-turn, the user typed:*“Why do you keep fighting with tools calls? Why not just scp the files locally and then scp it back?”Two minutes later Kimi’s thinking answers it:“That’s… exactly what I’m doing? The friction was (1) CRLF line endings breaking exact-string Edit matches, (2) my guessed anchor text not matching the actual file… Better: read the actual local file first, then edit with exact text.”*It changed method and the edits stopped failing. That is a mid-turn steer absorbed correctly without a restart — the one piece of technical direction it got, and it landed.

Two failures were not Kimi’s fault and should be discounted from any judgement of the model:

13 minutes dead to a protocol incompatibility. Between 14:16 and 14:29 the endpoint returned400 Invalid request: the message at position 512 with role 'assistant' must not be empty

five times. The transcript shows the user cycling/exit

→continue

five times to get past it. Six of the nine “continue” prompts in this session are that recovery loop, not the model stalling.The budget ended the session mid-edit. First429 · account suspended due to insufficient balance

at 14:44, terminal at 15:14. Kimi’s last action was anEdit

at 15:11:11 fixing aC2065 forward-reference it had just correctly diagnosed. It never got to compile, let alone test, its own redesign.

Efficiency verdict. Per round trip Kimi was cheap — smallest context, fewest tokens per tool call. What it lacked was

yield: 596 output tokens per response of which roughly two-thirds was internal reasoning, 7 K characters of prose to the user in three hours, and a tool-failure rate that turned a meaningful slice of the budget into recovery. It was not profligate; it was slow to convert tokens into verified state.

This was my subjective experience as well. Kimi K3 spent most of its tokens fighting the tooling but when it got past the hurdle, the analysis it made felt smarter than Opus 4.8.

Fable 5 on the same prompt, at fractions of the same budget

The interesting numbers aren’t at the iso-token line — they’re well before it.

Milestone	Wall clock	Responses	Billed	% of Kimi’s budget	List cost
Full audit delivered — two root causes, both proven live with its own instrumentation, MSDN contract points verified, diagnostics patch + README written, memory updated	46 min	78	14.4 M	20%	$22
Both fixes implemented, built, committed, and validated — GT1 ×3 at 47.2/46.2/48.7 fps, tear test 0/20, overlap ≈1.0, timeouts=0 . Then it stopped and idled for five hours.
1 h 57 m	197	57.1 M	80%	$73
Iso-token line — by now three hours deep into a second problem (fullscreen scanout going headless) with the user watching the screen
7 h 20 m	228	71.5 M	100%	$94
Session end — UMD run-ahead pacing, QEMU cocoa layer fixes, virglrenderer fence publication, an ETW investigation that falsified its own theory, a full Fire Strike matrix on both backends, and a history rewrite	17 h 54 m	611	322.0 M	451%	$392

20% of Kimi’s token budget, and a committed, benchmark-validated fix on

80%. Then it went idle for five hours because it had finished what it was asked to do — which is itself a signal worth noting next to Kimi’s nine “continue” prompts.

The fairness ledger

Four things separate these two runs besides the model. Two favour Fable heavily.

Fable started with Kimi’s conclusion in its context. This is the big one. Claude Code auto-loadsMEMORY.md

at session start, and the top line at 00:20 on Jul 23 read:*“Present-gate token association —*That line is Kimi’s + Opus’s output from the previous night, and Fable’s first fix is precisely the design it names. In Fable’s favour: it re-derived the mechanism from code and proved it live with its own instrumentation, and only opened the full memory file at 01:04 —FIXED— black frames = per-process token table exhausted after 16 pids →tok=0

UN-GATED flips;fix = adapter-global monotonic token. 2 DEAD ENDS: per-device stamp = bugcheck 0xFC, contiguous retire = permanent wedge.”afterthat live proof, at which point it corrected the note from “FIXED” to “BROKEN AT HEAD”. But it was not searching a blank space.Kimi inherited a dirty tree; Fable a clean one. Kimi opened on HEAD plus 736 uncommitted lines of a previous agent’s in-progress work, and spent real effort establishing that the working diff matched the savedUNSPLIT

patch. Opus split and committed that work overnight at the user’s request, so Fable opened on a clean tree at the same two commits.Kimi lost 7% of its span to a client/API incompatibility unrelated to reasoning quality, and was killed mid-edit by the balance cap.Fable found something no one had handed it. The FlipThread vsync-starvation bug appears in no handoff, no memory file, and no prior session. Fable found it in its first 46 minutes by adding its own vsync counters, and its fix (periodicKTIMER

+KeWaitForMultipleObjects

) survives in the tree today. Kimi had the same counters available and never looked at vsync cadence — that gap is not explained by the head start.

Both ran at effort max

with a 1 M context window. Kimi ran through an Anthropic-compatible third-party endpoint with tool search disabled; its context peaked around 404 K, Fable’s around 997 K before compaction.

The day before: Opus 4.8, and what “a lot of hand-holding” cost

This is the session that produced the ground both later runs stood on — and it is also the least autonomous of the three by a wide margin. 22.9 hours, 533 M tokens, 1,128 responses, 92 separate human or harness interventions, five gate architectures abandoned and three further theories refuted.

Composition of interventions

/goal

enforcement auto-compaction The raw rate is nearly identical across all three — 6.6, 6.0 and 6.6 interventions per active hour for Opus, Fable and Kimi. What differs completely is the kind:

Opus — 92, of which 55 substantive. Roughly19 were corrections of a wrong claim or a wrong path, on top of 17 interrupts and 16 Stop-hook messages generated by two/goal

conditions the user set specifically to stop it giving up and reverting. Median unattended run:8 responses. Eleven of the 55 were typed mid-turn and appear only in the queue records.** Fable — 34, of which 26 substantive.About 9 were human eyes-on-screen reports the model genuinely could not obtain itself, 6 were git hygiene requests, and 6 were design challenges — every one of which produced a real result. One bare nudge in eighteen hours. Median unattended run:15 responses; longest, 118 over 63 minutes. Kimi — 19, of which only 5 substantive:**the opening prompt, a progress-report request, a debugging hint (“check kdnet, the kmd likely crashed”), the mid-turn workflow question, and the closing cleanup request.Ten of the nineteen are bare nudges— ninecontinue

and one/compact

, and six of those nine are the recovery loop for the API-400 fault rather than the model stalling. So: two pieces of technical direction in three hours. Median unattended run:18 responses; it ran 156 responses over 104 minutes before the first human word.

Autonomy, measured three ways

“Can it operate autonomously” resolves into three separable questions: how long does it run unattended, how often does it need correcting as opposed to informing, and does it self-arrest when it has finished.

• Median unattended runKimi 18
• Fable 15
• Opus 8

**156**(104 min) · Fable

**118**(63 min) · Opus

**220**(180 min).

• Corrective inputs / active hourOpus 3.7
• Fable 1.05
• Kimi 0.69

ratedoes not.

does it go down wrong paths or zone in on the solution?:

Kimi took the fewest wrong turns per unit of work and needed almost no steering; it simply converted tokens into verified state slowly, and ran out.

Opus took the most wrong turns by a large margin and needed a human to catch three of them — but it was also the only session doing genuinely open-ended search, with no handoff describing the answer.

Fable is the one that both zoned in and finished, which is exactly what you’d expect from the run that inherited two days of someone else’s conclusions.

Verdicts

Kimi K3

Right direction · slow conversion · killed early

Direction: yes. It identified the correct root-cause class within 105 minutes, found the real defect in the incumbent fix, eliminated one candidate design with a kernel crash dump, and proposed the next one — which was good enough that both Opus and Fable adopted it before Fable found the flaw. Six of its ten substantive claims survive unchallenged.

Efficiency: mixed. Leanest context and cheapest per tool call of the three, but the lowest yield per token — two-thirds of its output was internal reasoning, its tool-failure rate was 8.0% against Fable’s 4.3%, and roughly a fifth of its wall clock went to environment friction it partly created (CRLF anchors, header hunting, repeated failed greps). When the user pointed that out mid-turn it diagnosed the cause correctly and changed method.

The real miss isn’t the superseded design — it’s declaring the multi-second frozen frame benign. That was failure mode 2 in the prompt, it had two genuine driver causes, and Fable found the first one in 46 minutes using instrumentation Kimi had already built.

Fable 5

Zoned in · finished · inherited a head start

Delivered a complete two-root-cause audit on 20% of Kimi’s budget and a committed, GT1-validated fix on 80%. Then stopped, having done what it was asked. Over the following 15 hours it fixed the scanout path, falsified its own disk-burst theory with a guest ETW trace and said so plainly, ran a controlled experiment on a UMD threading cap and rejected it, reverted a design it had shipped hours earlier once it proved it wrong, and rewrote history to remove it — verifying the final tree bit-identical to the benchmarked one.

Caveat that matters: its opening context contained Kimi’s and Opus’s conclusion in one line, including the fix design and both dead ends. The iso-token comparison is not a cold-start comparison. Its independent contribution is the vsync-starvation find, the per-thread association, the flip queue, and the UMD pacing — none of which was handed to it.

Opus 4.8

Hardest problem · most steering · foundational output

The only session doing open-ended search with no handoff describing the answer, and the one that paid for it: eight abandoned architectures, three claims retracted only after the user challenged them, 17 interrupts, and two /goal

Stop-hook conditions written specifically to stop it giving up and reverting.

It also found the bug that explained why every gate design had failed — the recycled manual-reset proxy fd — and shipped four commits across four repos plus the entire measurement apparatus the other two sessions used. Its 21-minute rescue of Kimi’s broken tree is the sharpest work in the whole dataset: diagnose a bugcheck, fix it, refuse a passing measurement, prove the underlying design unsound, replace it, re-measure, hand off.

The honest reading is that heavy hand-holding was working: every correction landed and moved the investigation forward. It just needed a lot of them.

source & further reading

blog.getutm.app — original article Quick tips for fast iteration in Haskell