{"slug": "understanding-is-the-bottleneck-why-agent-generated-code-needs-structured", "title": "Understanding Is the Bottleneck: Why Agent-Generated Code Needs Structured Summaries", "summary": "Geoffrey Litt's talk 'Understanding Is the New Bottleneck' argues that AI makes writing code cheap but understanding it remains costly, a problem amplified in agent-assisted development where code review often becomes rubber-stamping. Developer Adl Alternative built the open-source tool 'what-you-did' to address this by generating structured JSON summaries of code changes, enabling both human triage and machine processing for MR review.", "body_md": "# Understanding Is the Bottleneck: Why Agent-Generated Code Needs Structured Summaries\n\nGeoffrey Litt recently gave a talk called * Understanding Is the New Bottleneck*. His core point: AI makes writing code extremely cheap, but understanding code costs the same — or even more.\n\nI deeply agree. And in the context of Agent-assisted development, this problem is far worse than he describes.\n\n## The status quo\n\nAgent writes code for you. You run tests. Green. Open MR. Reviewer sees 47 files changed, 3000 lines. Spends 5 minutes skimming. Approve.\n\nNobody actually understands what the code does.\n\nThis isn’t hypothetical — it’s daily reality. Tests become the only quality gate, and code review degrades into rubber-stamping.\n\n## Why this is dangerous\n\nTests cover the scenarios you can think of. The problems Agent introduces are often where you can’t: unreasonable abstractions, hidden state coupling, performance regressions, missed edge cases. Only a human who understands the code logic can catch these.\n\nPut differently: tests tell you “it’s not broken now,” but not “will it break in the future.”\n\n## Where Geoffrey and I diverge\n\nGeoffrey’s answer is *educational*: he builds `/explain-diff`\n\n— rich HTML explainer docs with background context, literate diffs, interactive figures, and quizzes. His goal is to help humans *learn* the code deeply enough to participate in the creative process.\n\nI love this direction. But I’m solving a slightly different problem: **what happens before a human decides to read?** When a reviewer opens an MR with 47 files, they need triage — which files matter, what’s the shape of this change, where should I focus? They need a\n\n*map*before they start\n\n*exploring*.\n\nGeoffrey’s explainer is the exploration. I’m building the map.\n\n## Why unstructured explanations aren’t enough\n\nI tried having Agent explain its changes in natural language. Problems:\n\n**Unreliable**: AI might miss files, or over-explain trivial changes** Unconsumable**: Natural language can’t be processed by downstream tools (MR templates, impact analysis, change classification)** Unverifiable**: You can’t assert “the explanation covers all changed files”\n\nFundamentally, an essay-style explanation is a *destination format*. What I need is an *intermediate data format* — something both humans and machines can consume.\n\n## The structured output approach\n\nSo I built [/what-you-did](https://github.com/adlternative/what-you-did). Core design:\n\n- Use\n`git diff --name-status`\n\nas ground truth — build a skeleton containing every changed file first - Fill summaries per-file — entries can only be filled, never added or removed\n- Output JSON with a\n`file_count`\n\nfield for completeness verification - Aggregate by directory — provide different granularity levels for understanding\n\n```\n{\n  \"file_count\": 5,\n  \"summary\": \"Refactored cache module into L1/L2 dual-layer architecture\",\n  \"directories\": [\n    { \"path\": \"src/cache\", \"file_count\": 3, \"summary\": \"Cache core logic split\" }\n  ],\n  \"files\": [\n    { \"path\": \"src/cache/l1.ts\", \"status\": \"A\", \"category\": \"feature\", \"summary\": \"In-memory cache implementation\" },\n    { \"path\": \"src/cache/l2.ts\", \"status\": \"A\", \"category\": \"feature\", \"summary\": \"Disk cache implementation\" },\n    { \"path\": \"src/cache/index.ts\", \"status\": \"M\", \"category\": \"refactor\", \"summary\": \"Unified entry, routes to L1/L2\" }\n  ]\n}\n```\n\nThis isn’t prose for humans to read — it’s structured data for toolchains to consume. Humans can scan the summary at a glance; tools can auto-generate MR descriptions, trigger extra checks for specific modules, or compute change statistics by category.\n\n## What this enables: MR review integration\n\nStructured data can be rendered into any shape. Here’s a concept for MR review — hover any file or directory to see its AI summary:\n\nThe key insight: this data is available *at page load time*. No extra API calls. The JSON was generated when the branch was pushed — rendering is just a frontend component away.\n\n## Key design decisions\n\n**Skeleton first**: Get the complete file list from git before filling content. This guarantees no files are missed. “Let AI analyze then output” has no such guarantee — especially on large diffs where context windows get tight.\n\n**JSON not Markdown**: Markdown is a rendering format. JSON is a data format. One JSON file can become an MR comment, a Slack notification, a weekly report entry, a changelog — each consuming what they need.\n\n**Directory-level aggregation**: Nobody wants to review 47 files one by one. But “8 files changed under src/auth, overall adding OAuth2 support” gives instant judgment.\n\n## The relationship to Geoffrey’s work\n\nI see our approaches as complementary layers:\n\n**Structured summary**(what-you-did) → triage, orient, decide where to focus** Rich explanation**(explain-diff) → deep understanding of the parts that matter** Quiz/verification**→ confirm you actually understood\n\nThe first layer is fast, mechanical, and verifiable. The second is rich, educational, and creative. You need both.\n\n## Conclusion\n\nThe cost of writing code is approaching zero, but the cost of understanding code hasn’t changed. We need tools at multiple levels to bridge this gap — structured data for the map, rich explanations for the territory.\n\n*Project: what-you-did on GitHub*\n\nGeoffrey Litt 最近做了一个演讲叫 * Understanding Is the New Bottleneck*，核心观点是：AI 让写代码变得极其廉价，但理解代码的成本没有变——甚至更高了。\n\n我深有同感，而且在 Agent 辅助开发的场景下，这个问题比他描述的更严重。\n\n## 现状\n\nAgent 帮你写完代码，你跑测试，绿了，提 MR。评审人点开 diff，看到 47 个文件变更，3000 行代码。他花 5 分钟扫了一眼，approve。\n\n没人真的理解这些代码做了什么。\n\n这不是假设，这是我观察到的日常。测试通过成了唯一的质量门禁，代码评审退化成了形式审批。\n\n## 为什么危险\n\n测试覆盖的是你能想到的场景。Agent 引入的问题往往在你想不到的地方：不合理的抽象、隐含的状态耦合、性能退化、边界条件遗漏。这些只有人类理解代码逻辑后才能发现。\n\n换句话说：测试告诉你”现在没坏”，但不告诉你”未来会不会坏”。\n\n## 我和 Geoffrey 的分歧\n\nGeoffrey 的答案是*教育导向*的：他做了 `/explain-diff`\n\n——生成包含背景知识、文学化 diff、交互图示和测验的 HTML 解释文档。目标是帮助人类*深入学习*代码，从而能参与创造性过程。\n\n我很喜欢这个方向。但我在解决一个稍微不同的问题：**在人类决定深入阅读之前会发生什么？** 当评审者打开一个 47 文件的 MR，他首先需要分诊——哪些文件重要、变更的整体形状是什么、应该聚焦在哪里？他需要先有一张*地图*，然后才能开始*探索*。\n\nGeoffrey 的解释器是探索工具。我在做地图。\n\n## 为什么非结构化的解释不够\n\n我试过让 Agent 用自然语言解释它做了什么。问题是：\n\n**不可靠**：AI 可能漏掉文件，也可能过度解释无关紧要的改动**不可消费**：自然语言没法被下游工具处理（MR 模板、影响分析、变更分类）**不可验证**：你没法断言”解释覆盖了所有变更文件”\n\n本质上，散文式解释是*终态格式*。我需要的是*中间数据格式*——人和机器都能消费的东西。\n\n## 结构化输出的思路\n\n所以我做了 [/what-you-did](https://github.com/adlternative/what-you-did)。核心设计：\n\n- 用\n`git diff --name-status`\n\n作为 ground truth，先建一个包含所有变更文件的骨架 - 逐文件填充摘要，不允许增删条目\n- 输出 JSON，带\n`file_count`\n\n字段用于校验完整性 - 按目录聚合，提供不同粒度的理解入口\n\n```\n{\n  \"file_count\": 5,\n  \"summary\": \"重构缓存模块为 L1/L2 双层架构\",\n  \"directories\": [\n    { \"path\": \"src/cache\", \"file_count\": 3, \"summary\": \"缓存核心逻辑拆分\" }\n  ],\n  \"files\": [\n    { \"path\": \"src/cache/l1.ts\", \"status\": \"A\", \"category\": \"feature\", \"summary\": \"内存缓存实现\" },\n    { \"path\": \"src/cache/l2.ts\", \"status\": \"A\", \"category\": \"feature\", \"summary\": \"磁盘缓存实现\" },\n    { \"path\": \"src/cache/index.ts\", \"status\": \"M\", \"category\": \"refactor\", \"summary\": \"统一入口，路由到 L1/L2\" }\n  ]\n}\n```\n\n这不是给人读的散文，是给工具链消费的结构化数据。人可以快速扫一眼 summary，工具可以自动生成 MR 描述、触发特定模块的额外检查、或者按 category 做变更统计。\n\n## 下游怎么用：MR 评审界面集成\n\n结构化数据可以被渲染成任何形态。这是一个 MR 评审的概念——hover 文件或目录查看 AI 摘要：\n\n关键点：这些数据在页面加载时就已经有了。零额外 API 请求。JSON 在分支 push 时就生成了——渲染只是一个前端组件的事情。\n\n## 关键设计决策\n\n**骨架先行**：先从 git 拿到完整文件列表，再填充内容。保证不会漏文件。传统的”让 AI 分析 diff 然后输出”没有这个保证——尤其是大 diff 接近 context window 极限时。\n\n**JSON 而非 Markdown**：Markdown 是渲染格式，JSON 是数据格式。一份 JSON 可以变成 MR 评论、Slack 通知、周报条目、变更日志——各取所需。\n\n**目录级聚合**：47 个文件没人想逐个看。但”src/auth 下改了 8 个文件，整体是在做 OAuth2 支持”——立刻就有判断力。\n\n## 和 Geoffrey 工作的关系\n\n我认为我们的方案是互补的层次：\n\n**结构化摘要**（what-you-did）→ 分诊、定向、决定聚焦点**深度解释**（explain-diff）→ 对重要部分的深入理解**测验/验证**→ 确认你真的理解了\n\n第一层快速、机械、可验证。第二层丰富、教育性、有创造力。两者都需要。\n\n## 总结\n\n写代码的成本在趋近于零，但理解代码的成本没变。我们需要多层次的工具来弥合这个差距——结构化数据做地图，深度解释做探索。", "url": "https://wpnews.pro/news/understanding-is-the-bottleneck-why-agent-generated-code-needs-structured", "canonical_source": "https://adlternative.github.io/posts/ai/understanding-is-the-bottleneck/", "published_at": "2026-08-16 16:12:04.640232+00:00", "updated_at": "2026-08-16 16:12:06.396296+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "artificial-intelligence"], "entities": ["Geoffrey Litt", "what-you-did", "Adl Alternative"], "alternates": {"html": "https://wpnews.pro/news/understanding-is-the-bottleneck-why-agent-generated-code-needs-structured", "markdown": "https://wpnews.pro/news/understanding-is-the-bottleneck-why-agent-generated-code-needs-structured.md", "text": "https://wpnews.pro/news/understanding-is-the-bottleneck-why-agent-generated-code-needs-structured.txt", "jsonld": "https://wpnews.pro/news/understanding-is-the-bottleneck-why-agent-generated-code-needs-structured.jsonld"}}