# The wrong question when an AI breaks is "where"; the right one is "which layer" > Source: > Published: 2026-09-27 04:44:00+00:00 *AI Harness Engineering · Essay Ten · derek wang (derekwang85)* Hemingway said a story works the way an iceberg does — only one-eighth floats above the water, and the seven-eighths below is what actually carries the weight. He meant it about writing, but it is the truest thing I know about debugging AI work, and about what it means to *harness* one: steering a model's force while restraining where that force is allowed to land is, more than anything, a skill for reading what sits below the surface. When an AI produces something broken, the first thing you see is the phenomenon. A paragraph of blather. A file that won't compile. Numbers that don't line up. An API that timed out. You open the logs, see red, and by reflex ask the single most dangerous question in engineering: **"Where is this wrong?"** That question is the trap. It presumes "wrong" is a concrete spot you can click — a function, a line, a field. So you click down the stack until something looks suspicious and declare, "found it." Almost always, what you clicked is just the eighth that floats. The real cause sits below, and the question misdirected you from the moment it left your mouth. We split that "below" into three clean depths of *why*. **L1, the phenomenon.** The error itself. It describes *what happened* and almost never *why*. "The interface timed out" is L1. Practically every bug report stops here, because it feels like an answer. **L2, the structure.** Behind the phenomenon, is the code's own shape wrong? The L2 of "interface timeout" might be *the service never sets a timeout configuration and the default is absurdly small.* The phenomenon tells you the result; the structure tells you what is wrong with the layout. **L3, the design.** Deeper still, is the plan itself flawed? The design problem behind "the service never set a timeout" might be *we never designed for "another service could be slow" at all.* That is not a field problem; it is an assumption problem. These three layers are not an academic taxonomy. We made them a hard gate: **every failure must complete the three-layer iceberg analysis before it may be fixed.** And at the center of that gate sits one rule aimed straight at the heart of most debugging disasters: **It is forbidden to call "a data problem" a root cause.** Those four words hide a mountain of ruined afternoons. Watch a team chase a bug for three days and conclude, at last, "oh, the data is just wrong." It sounds airtight — but that conclusion is only L1's phenomenon restated in a lab coat. Why is the data wrong? Did the upstream feed it bad data, did validation miss it, or is the format convention inconsistent? Behind every "data problem" sits a structure problem and a design problem. Letting "data problem" stand as root cause is how a team grants itself a permanent parking place on top of the iceberg. But *willing to dig* is only the first lesson. The sharper danger arrives the second you believe "root cause found, time to code" — because most AI incidents do not die from never finding the cause. They die from **fixing this one spot and waking up something else.** Engineering failures are rarely one wrong spot. They are *one change that detonates another.* Early on we practiced what I now call head-medicine, because it happened in the head: the data was bad, treat the data. We changed the field, the unpack worked, the regression passed. Days later, a completely unrelated module started throwing. It took a long search to find that last week's "successful" fix had quietly altered a corner nobody had thought to check. So a rule emerged that sounds like a platitude and is brutally hard to follow: **before you change A, think about B C D E.** We mechanized it into something called ripple analysis. Before touching a spot you scan its blast radius — direct and indirect dependencies, who references it, what it references — then cascade regression outward in rings: a change that stays inside one file gets the shallowest ring; one that reaches related modules gets the next; one that climbs a whole chain gets another; and a change that touches a convention, a contract, or an interface gets **full regression.** That R0-to-R4 ladder exists for exactly one reason: before "my change A" ever ships, sweep through every B, C, D, and E it could disturb and test each one. Changing one line and dragging ten along with it is not inefficiency. It is correctness. A real scar. A bug we chased in three rounds. Round one: the agent read the error, decided it was a wrong field unpack, changed it — and the regression only half-passed. Round two: looking closer, the field and the unpack were both actually correct. The real culprit was **another microservice's interface timeout.** It reached the structure layer, but not the bottom. Round three: the true problem surfaced — the remote call **carried no timeout parameter at all,** the default was too small, and any slowness blew it up. At that depth automatic repair could not carry it, because it implicates a cross-service decision: whether to add configuration to the dependency. That is when a human had to step in and make the call. The lesson is not that the agent was smart. It is that **it almost declared the job done in round one.** Without the three-layer gate forcing it to keep digging, without ripple regression forcing it to map the full blast radius, it would have stopped at a false root cause and left a time bomb behind. Most teams, most of the time, stop at round one. The craft of debugging is not how fast you find one bug; it is whether you will read the seven-eighths below the water. A diagnosis that stays at the surface is pain relief; a fix that reaches the root cause is a cure. You can fool a colleague and you can even fool yourself — but an incident has no manners. Two small mechanisms guard attribution. First, **REASONING_BLOCK**: every commit carries the agent's reasoning chain — how it traveled from phenomenon to root cause — so anyone revisiting the change later sees the road the attribution took, not just the conclusion. Second, the **Issue Log runs on three powers that must never be one agent**: the one who reports a bug, the one who fixes it, and the one who accepts it. Every issue carries a three-layer analysis and a fix trail, and anything stalled for more than two days auto-escalates to P0. It is not process for the sake of prettiness. It forces the attribution to be re-validated a second time, through a third set of eyes. And this is where memory — Essay Nine's subject — stops being decoration and becomes the whole point. When a new issue arrives, the Worker does not bury its head in analysis first. It queries derekinside: **has this kind of problem been handled before?** If yes, pulls out the old solution immediately — how it was characterized, how it was fixed, what notes were left. That turns causation from repeated chore into compounding speed. The same class of bug takes an hour the first time, ten minutes the second, and on the third you localize it with a glance. Memory's final value is not that you remember more. It is that you attribute faster and more accurately. Cold water, because "three-layer iceberg" and "ripple regression" sound so good that they beg to be abused. Not every bug deserves the full treatment. A typo, a wrong fixed-copy string, an obvious slip of the pen — demanding design-layer root-cause analysis for those is not rigor, it is fuss. An over-attributed system, like an over-tendered company, drags itself to a standstill under its own process. The judgment has one test: **if I stop at this layer, will it come back?** If fixing the phenomenon keeps it from ever recurring, it was a phenomenon-level problem. If it changes shape and returns next month, it has earned another dive. The iceberg's job is to stop you from staring at the surface alone — not to force a deep-sea dive on every ripple.