{"slug": "what-if-coding-agents-didn-t-have-to-read-code", "title": "What if coding agents didn't have to read code?", "summary": "Benzi, a coding agent that compiles codebases into a resolved map instead of reading raw files, achieved a 45.5% pass rate on SWE-bench Verified (500 instances, one attempt each) using DeepSeek v4-flash, with network access blocked to prevent answer lookup. In a 24-bug cross-harness benchmark, Benzi read 9,125 lines with Sonnet and 16,407 with DeepSeek, compared to 20,704 for Claude Code with Sonnet and 43,598 for DeepSeek Harness, demonstrating up to 4.8× fewer lines read.", "body_md": "Every other agent dumps your files into a context window and hopes. Benzi compiles your codebase into a resolved map first — calls, data flow, references — then navigates it with real tools. One compiler, ten languages, one map.\n\nSWE-bench Verified · 500 instances · one attempt each\n\n`swebench.harness.run_evaluation`\n\nThe full SWE-bench Verified set — 500 real GitHub issues from twelve Python\nrepositories — run end to end through Benzi on **DeepSeek v4-flash**, graded by the\nofficial SWE-bench harness inside its own per-instance Docker images. Network access to GitHub\nand PyPI was blocked inside every container, so nothing could look up an answer.\n\nTry Benzi\n\nTwo capabilities, side by side. On the left, Benzi taking apart a 30-year-old engine. On the right, a whole app it built from a single chat.\n\nNo summaries from memory — the actual source, resolved and explained.\n\nEveryone says DOOM’s engine was ahead of its time. Almost nobody has\nopened `z_zone.c`\n\nto see why. So Benzi did. A few things were worth writing down.\n\n`malloc()`\n\nduring gameplay.id wrote their own memory allocator — one big arena grabbed once at startup, sliced\ninto blocks tagged by how precious they are (`PU_STATIC`\n\n, `PU_LEVEL`\n\n,\n`PU_CACHE`\n\n…). The genius part: allocating new memory can silently evict old\n“cache” blocks it walks past along the way — no one calls `free()`\n\n,\nthe allocator just decides your cached texture is cheap to regenerate and reclaims the space\non the spot. That’s cache-eviction policy baked directly into the allocation path itself.\n`malloc`\n\n/`free`\n\nstill can’t do that today.\n\n`tables.c`\n\nis a 2,000+ line file that is almost entirely one thing: every sine,\ntangent and arctangent value the engine will ever need, precomputed at compile time into\nlookup tables. Movement, angles, rendering — all fixed-point integer math against these\ntables. Not every ’93 machine had an FPU, and even where it did, table lookups beat live\ntrig every time.\n\n`screens[0]`\n\nis a flat 320×200 buffer, one byte per pixel. The 3D world gets\ndrawn into it column by column. Then the HUD gets stamped on top using the exact same\npixel-blitting function used to draw monster sprites and gun sprites. There is no UI toolkit,\nno widget tree, because there was nothing to build one on top of: the game owns the entire\ndisplay, full stop. A health digit and a demon sprite are the same kind of draw call.\n\n`p_maputl.c`\n\nsplits the map into a grid (the “blockmap”) so hit\ndetection only checks nearby geometry instead of scanning every wall in the level — a\nspatial hash, built from scratch, years before that was a common technique people talked about.\n\nNone of this was over-engineering. Every one of these systems exists because the\nstandard answer (**malloc, floats, a GUI library, brute-force collision**) either didn’t\nexist on the target hardware or would have been too slow — and Benzi surfaced each one by\n**resolving the actual code**, not guessing from what it remembered about DOOM.\n\nA dating app for horses, greenfielded in one chat session.\n\nProcedural SVG portraits (no image is a\nfile — every horse is drawn in code), a swipe deck, and **live AI chat where every match\nflirts back** through a real model. Frontend, backend, and the prompts — all written by Benzi.\n\n24-bug cross-harness benchmark · each point is one bug\n\n| Harness · model | Lines read | vs Benzi |\n|---|---|---|\n| Benzi · Sonnet | 9,125 | — |\n| Benzi · DeepSeek | 16,407 | 1.8× |\n| Claude Code · Sonnet | 20,704 | 2.3× |\n| DeepSeek Harness · DeepSeek | 43,598 | 4.8× |\n\n**Lines read** counts only what came back from\nfile-read calls — grep and shell output are search, not reading. It is the one\nfigure that means the same thing in every harness, which is why it is the one\ncompared here.\n\nFrom the 24-bug cross-harness benchmark: every harness opens more source as bugs get harder — the question is the slope. Each point is one bug, laid out easiest to hardest, left to right. Hover any point for the bug and its count.\n\nDifficulty is Claude Code's turn count on that bug — a third-party yardstick, so no\nharness sets its own position on the axis. **Lines read** counts only what came back from\nfile-read calls; grep and shell output are search, not reading. The figure beside each line is its\nslope: how many extra lines that harness opens per step of difficulty.\n\nThe same 24 bugs in the same order, with wall clock in place of lines read.\n\nWall clock is **raw** — Benzi's per-repo index build is not subtracted.\nEach point is that harness's most recent solved run for that bug; unsolved and unfinished runs are\nleft out rather than plotted as fast. Benzi on Sonnet never solved http-parser and the DeepSeek\nharness never ran nats-server, so those two points are absent and neither enters its fit.\n\nAnd the same again with dollars on the vertical axis.\n\nPriced at the published per-token rates, same run selection as the chart above. The\ntwo DeepSeek series run an order of magnitude cheaper than the two Sonnet ones, so at this scale they\nsit close to the baseline. What the axis does show is the **slope**: Claude Code's cost climbs\nwith difficulty faster than any other series here. Full tables behind every point live on the\n[benchmark page](https://benzi.fly.dev/benchmark).\n\nThe architecture · one compiler, one agent loop\n\nEverything that falls out of actually resolving the code — from the index itself to the gates on every write.\n\nEvery file parsed, imports resolved, class ancestry built, every identifier traced to its definition — before a single question is answered. Call flow and data flow are joined at every call site, so a bad value traces to its origin in one tool call. Claude Code greps; Cursor embeds; Benzi resolves — and answers in O(1).\n\nEvery write passes syntax and semantic gates against the real language parser — a broken parse auto-reverts. Every write that lands reports its blast radius: the changed symbol, its callers, its holders — and the same analysis pulls in the selectively relevant existing tests.\n\nProven edges carry evidence. Ambiguous calls keep their candidate lists instead of a guess. Runtime traces settle what static analysis can't.\n\nEvery unresolved call is classified: a library call, an in-repo call with recorded candidates, or an honest unknown with the ID the compiler supposed.\n\nFollows actual execution through the compiler's map — real arguments, real returns, real dispatch — overlaid back onto the static index.\n\nA focused, model-generated repro against the exact change, run under the tracer to fast-track testing. No mock harness.\n\nThe map that drives the tools drives the live call graph beside the chat. When Benzi names a function, that node lights up.\n\nA separate index for HTML/CSS/DOM-JS — cascade resolution, selector specificity, JS grabs, even frontend inside Python strings.\n\nDurable per-repo facts survive restarts — conventions learned once aren't re-derived every session, and wrong memories get deleted, not hoarded.\n\nAnthropic, OpenAI, or any compatible API — and when a task outgrows the model running it, Benzi escalates itself to a bigger one mid-task.\n\nLinks · everywhere Benzi lives", "url": "https://wpnews.pro/news/what-if-coding-agents-didn-t-have-to-read-code", "canonical_source": "https://benzi.fly.dev/about", "published_at": "2026-08-23 10:01:03+00:00", "updated_at": "2026-08-23 10:13:50.682153+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools", "machine-learning"], "entities": ["Benzi", "DeepSeek v4-flash", "SWE-bench Verified", "Claude Code", "Sonnet", "DeepSeek"], "alternates": {"html": "https://wpnews.pro/news/what-if-coding-agents-didn-t-have-to-read-code", "markdown": "https://wpnews.pro/news/what-if-coding-agents-didn-t-have-to-read-code.md", "text": "https://wpnews.pro/news/what-if-coding-agents-didn-t-have-to-read-code.txt", "jsonld": "https://wpnews.pro/news/what-if-coding-agents-didn-t-have-to-read-code.jsonld"}}