We Made Bug Tickets Write Their Own Root Cause Analysis. The Value Wasn't the RCAs — It Was What They Became in Aggregate.
🚀 Every bugfix now ships with a structured, machine-written root cause analysis, authored at fix time by a coding agent. It took under two days to build. Reading the last hundred of them back is what changed how we plan quality work.
#
The Problem
We wanted to answer a set of questions about our own codebase that we could not answer:
Which areas break repeatedly? Not "where are the bugs" — every team can guess that. Which modules keep producing the same class of failure, normalized for how much they change. #
Are we getting better? We had no way to tell improvement from a quiet quarter. Bug count moves with traffic, headcount, and release cadence; it doesn't tell you whether the underlying quality changed. #
What kind of bugs are these, really? Process failures and architectural failures need completely different responses, and we had no categorization to separate them. #
Where are our detection gaps? Which gate should have caught each defect, and which missing gate would pay off most if we built it. #
What's actually worth refactoring? We wanted a heatmap of problematic areas grounded in defect history, not in whoever complained loudest in planning.
Every one of those is a query over a corpus. We had years of bug history and nothing queryable in it — because RCA lived in a free-text box, where it fills up with things like "the dropdown click is triggering the click-outside handler." True, useful to exactly one person for exactly one week, and impossible to aggregate.
The blocker was never willingness. It was that writing a real RCA costs an hour of archaeology at the exact moment someone is trying to close a ticket. So the fix isn't asking people to try harder — it's removing the moment where trying is required, and making the output structured enough to answer the questions above.
#
How we solved it
Author at fix time, not at ticket close. The agent runs when a dev starts a bugfix, while the spec, the git history, and the failing behavior are all still live context. By ticket close that context is gone, which is exactly why humans write one-liners. #
Seven fixed sections, not a text box. Summary · Cause Chain · Category · Feature · Component · Detection Gap · Prevention. Structure is the feature — free text can't be aggregated by construction. #
Cause chains carry citations, not adjectives. Each WHY-N links the PR and ticket that introduced that layer. This is the single design decision that mattered most: it turns an RCA from prose into a graph edge. #
Orthogonal categories. People / Process / Technology / Architecture, primary + secondary, each with a written justification. Lifted from ODC (IBM Research, early '90s — still the reference model for defect classification). #
Detection Gap is its own field. Found by
/ Should have been caught by
. A different question from "why did it break," with a different remediation budget. #
Prevention splits Quick Win vs Systemic. So the cheap fix doesn't quietly absorb the attention the architectural fix needed. #
Cost: under two days of work. It’s a low-barrier experiment, not a massive platform project.
#
What the field looks like
Shape of a real one, sanitized:
Look at WHY-4. A latent contract mismatch that had been sitting in the codebase for years, traced back to the exact PR that introduced it, with a named prevention. No engineer fixing a one-line schema bug was ever going to write that paragraph voluntarily — the effort/reward math doesn't work for a human. It works fine for an agent that already has the git history loaded.
#
What we got immediately
Depth nobody would have paid for by hand. Chains reach past the symptom to a process or design gap, and the cited ancestor PR is often years old. That archaeology is the expensive part, and it's the part that got automated. #
Consistency across teams. Two engineers on two services produce RCAs that parse with the same code. This is what makes downstream analytics possible. #
Detection Gap became a test backlog for free. Should have been caught by
, read across the corpus, is a list of the gates you don't have. Ours started saying "pre-merge contract checks" out loud, repeatedly, without anyone deciding that was the theme. #
*Reviewers get the *** why before they read the diff. The RCA lands on the ticket before the PR merges, shifting review focus from symptom to cause. #
Zero marginal cost per bug. The reason RCA programs die is that they tax the person least able to afford it — someone mid-fix, under pressure. Remove the tax and the practice survives on its own.
#
📈 What compounds over time
This is the actual payoff, and none of it works without structure. Given a corpus of these, you can derive:
Root-cause Pareto. Which handful of causes account for most of your defects. Old technique, still undefeated, previously impossible on free text. #
Component hotspot heatmap. Bug density per module normalized by churn — not "which file has bugs" but "which file has bugs for its size and change rate." The normalization is what makes it actionable instead of just pointing at your biggest file. #
Cross-domain blast radius. Pair (domain where the cause was introduced)
with (domain where the symptom surfaced)
. A Sankey of those pairs shows you where your module boundaries are lying to you. #
Detection-gap trend. Aggregated Should have been caught by
is your quality-gate investment plan, ranked by how often each missing gate would have paid off. It replaces annual budget debates with an empirical list. #
Regression genealogy. Follow WHY-N citations backwards to find PRs that keep reappearing as ancestors of later bugs. Change-failure attribution at the PR level, derived rather than guessed. #
Latency-to-detection distribution. How long defects sit before anyone notices, per area. Turns "we should improve observability" into "these three areas leak to customers and the rest don't." #
Category drift. A rising Architecture share is a different organizational signal than a rising Process share. Watching the mix move over quarters is far more informative than watching any total. #
Semantic recurrence detection. Embed cause chains; flag when a new bug's chain closely matches a closed one. The most actionable signal in the system, because a match means the prevention didn't take. #
Prevention follow-through. Did the Systemic
item ever get filed? Ever get done? The gap between quick wins shipped and systemic fixes shipped is technical debt accrual, made measurable.
#
Where it ended up
This started as one team's experiment — a single field, one squad's bugs, a skill written in under two days.
It didn't stay there. The field now lives on every ticket in Jira, org-wide, and that changed what the data is. A corpus scoped to one team answers questions about that team. A corpus spanning every domain answers questions no single team could ask:
Cross-domain cause tracing. Where a defect surfaces is often not where it was introduced. With per-team data, every cross-boundary cause looks like someone else's problem. #
Shared root causes. Four teams hitting the same class of failure is a platform problem wearing four disguises — visible only when their RCAs share a taxonomy. #
Comparable dashboards. A domain can see its own defect profile against the org baseline to recognize: "our detection gaps skew toward integration testing and nobody else's do." #
Org-level gate planning. Detection gaps aggregated across all domains rank which missing gate pays off most company-wide.
It spread for mundane reasons: cheap to build, no behavior change required, visibly useful before anyone had to be convinced. Nobody adopted a process — they adopted a field that was already filled in.
#
The uncomfortable parts
Machine-authored RCA has failure modes hand-written RCA doesn't. Naming them is part of the value:
Category distribution is not yet trustworthy. Ours skews heavily toward one category. Either the codebase really is like that, or the model has a prior toward the most intellectually satisfying explanation. Until checked against a human-labeled sample, treat category trend as signal and category level as a hypothesis. #
Citations need validating, not trusting. Links are only good news if they resolve. Citation resolvability is a metric to compute, not an assumption to make. #
Chains are context-bounded. A chain that stops early might mean the root cause was genuinely shallow, or that the model couldn’t see any further. The data does not distinguish those two. #
Prompt drift breaks trends. Change the skill and you've changed the measurement instrument. Version it and log it so anomalies in the trend line stay explainable. #
Never aggregate by author. Non-negotiable. The moment RCA data feeds performance reviews, the field becomes fiction and every downstream insight dies with it.
#
Takeaways
The bottleneck in RCA was never willingness — it was context cost. Move authoring to the moment context is free and the practice sustains itself. #
Structure is the whole product. Fixed sections are what let you ask questions of the corpus later. #
Citations turn documents into a graph. Everything genuinely novel in the analytics list above depends on the cause chain carrying links rather than adjectives. #
Build the data-quality view before the insight views. Otherwise you get beautiful charts sitting on unverified model output. Individual RCAs are the byproduct. The corpus is the asset.
P.S. Drop a comment or message me on LinkedIn if you'd like the exact prompt/skill definition we use to generate these structured RCA fields. Happy to share it!
#
SoftwareEngineering #SoftwareArchitecture #AIEngineering #DeveloperProductivity #DevOps