Running OpenWiki for Real: Why the Model You Pick Decides Everything A developer testing LangChain's OpenWiki tool on a Spring Boot/Angular monorepo found that Claude Sonnet repeatedly crashed due to malformed tool calls, while switching to Claude Opus 4.8 completed the full documentation run. The experience demonstrates that in agentic documentation tools, strict adherence to tool schemas matters more than general language quality. In Part 1 https://medium.com/towards-artificial-intelligence/openwiki-source-code-docs-that-write-and-maintain-themselves-a-hands-on-look-fcec781e28e4 I covered what OpenWiki is — LangChain’s take on Karpathy’s “LLM Wiki” idea, applied to codebases — and how trivially easy it is to install. This part is the reality check: what happened when I pointed it at an actual project, and the lesson that surprised me most. The short version: with an agentic doc tool, your choice of model isn’t a tuning knob — it’s the difference between a finished wiki and a crashed run. I aimed OpenWiki at a real monorepo — a Spring Boot 3.1 / Java 17 backend with an Angular 20 frontend . A few hundred source files, a genuine domain model, real business rules. Exactly the kind of codebase where good docs pay off and where “just read the code” doesn’t scale for a newcomer. I configured it against Anthropic and gave it a rich seed prompt describing the repo’s shape and domain, then let it run in one-shot --print mode so it would explore and write without me babysitting a chat loop. My first choice was Claude Sonnet — the sensible default for a “quality-but-not-premium” job. It connected fine, started exploring the codebase… and then the run died. Not with a network error or an out-of-tokens message. It died on a malformed tool call : the agent emitted a tool invocation that was missing a required field, the underlying framework rejected it against its schema, and — because this was a headless, non-interactive run — the whole process crashed instead of recovering. I re-ran it. It crashed again, at a different point, on a different tool. Once it was a file-writing call missing an argument; once it was a to-do-list call with entries missing a required field. The pattern was the tell: this wasn’t one unlucky bug. Sonnet was intermittently producing tool calls that didn’t conform to the strict schemas the agent framework demanded, and the headless run had no way to recover from it. This is worth sitting with, because it’s counter-intuitive. Sonnet is a strong model. It writes excellent prose. But OpenWiki isn’t a “write me a paragraph” task — under the hood it’s a demanding agentic tool-calling loop : dozens to hundreds of turns, each one issuing structured tool calls to read files, search, and write pages, sometimes fanning out to sub-agents. In that setting, the thing that matters most isn’t eloquence — it’s relentless, exact adherence to tool schemas, turn after turn . A model that slips even occasionally will eventually slip on the one call that brings the run down. Prompt-nudging didn’t save it, either. I added explicit instructions about tool usage; it then tripped on a tool I hadn’t even mentioned. You can’t prompt your way out of a model-reliability problem. So I switched the model to Claude Opus 4.8 model ID claude-opus-4-8 and changed nothing else about the task. It ran clean. It explored the repo, dispatched research sub-agents across the major domains, drew up a plan, and wrote the full doc set — architecture overview, authentication flow, the domain model, the REST API surface, a frontend guide, and an operations/runbook page — all cross-linked, all grounded in the current source. It even flagged that an older doc already in the repo was stale relative to the code, and pointedly refused to reproduce some credentials it found in a config file. Nine pages, coherent and accurate, on the first completed attempt. The single variable that changed was the model. Opus’s stronger adherence to structured tool calls was the entire difference between “crashes halfway” and “done.” For completeness: there were also a couple of early-version rough edges around how the tool reads certain file types — the kind of thing you’d expect from a v0.x release — which are already being addressed upstream. But those were papercuts. The model-reliability issue was the real story. If you take one thing from this: for OpenWiki — and agentic doc tools generally — pick your model for tool-calling reliability first, prose quality second. The failure mode isn’t “the docs come out a bit worse.” It’s “the run never finishes.” A cheaper or lighter model can be a false economy if it can’t complete the loop. Here’s the part nobody puts in the README. A good OpenWiki run isn’t one API call — it’s a long agentic session that reads much of your repo, reasons over it, spawns sub-agents, and writes several pages. That’s a lot of model calls, and the token bill is real. The Anthropic pricing that’s relevant 1 per 1 million tokens , at the time of writing : A few honest observations on cost: My rule of thumb: budget for a capable model on the initial generation where reliability and quality matter most and completion is non-negotiable , then let cheap, incremental --update runs carry the ongoing maintenance. That combination gives you a wiki that's both good and affordable to keep alive. Everything above is about the initial --init — the big one-time build. But the number that actually decides whether a tool like this is sustainable is the ongoing cost: what happens on the next commit, and the one after that? So I ran the experiment. Starting from the generated 9-page wiki, I made a small, realistic change — added one new duplicate-event endpoint, touching two source files a backend controller and the matching frontend service — committed it, and ran: openwiki --update Here’s exactly what it did: It was even sharp about the diff: it noticed the controller called a service method that wasn’t in the commit, and deliberately documented the endpoint at the API surface without inventing service-layer behavior that didn’t exist yet. That’s the diff-scoped reasoning working as intended. The cost implication is the whole point. --init explores hundreds of files, runs sub-agents, and writes every page — the expensive run. --update on a two-file commit read two files and rewrote two short sections. It is a tiny fraction of the initial cost, and it scales with the size of your diff, not the size of your repo . A one-line fix costs about what a one-line fix should; a big feature branch costs more, proportionally. This is the economic argument for the tool in one sentence: you pay real money once to generate, then pennies per commit to stay current — which is exactly the opposite of the usual doc lifecycle, where you pay nothing up front and then pay forever in staleness. A practical way to run it: wire openwiki --update into CI on merges to your main branch a GitHub Action ships with the repo . Each merge triggers a small, diff-scoped update, and your wiki tracks main automatically. Because updates are cheap and incremental, this is affordable to leave running indefinitely. One thing worth clearing up: OpenWiki doesn’t keep its own changelog of “this run touched these pages.” The only state it persists is a single openwiki/.last-update.json file — and that just records the latest run's metadata the commit it synced to, the model, a timestamp . It's a bookmark so the next --update knows where to start diffing, not a history. The history lives in git , by design. Because you commit the openwiki/ folder like any other source, every update's edits are captured as ordinary commits and diffs: git log --oneline -- openwiki/ every wiki change, run by rungit show