{"slug": "web-chat-is-not-a-coding-environment", "title": "Web chat is not a coding environment", "summary": "A developer argues that the gap between web chat interfaces like Claude.ai and ChatGPT and terminal-based coding harnesses is an interface gap, not a talent gap, because chat models generate text about a system they cannot see while harnesses run inside a runtime loop that can touch the actual repository. Citing LangChain's formulation that an agent equals a model plus a harness, the piece notes that changing only the harness moved a coding agent from outside the top 30 to top 5 on Terminal-Bench 2.0, and that Claude Opus 4.5 scored 42% on the standard CORE-Agent scaffold versus 78% on the Claude Code scaffold. The developer still uses web chat for architecture, rubber-ducking, and first drafts, but says it fails the moment output must land in a shared repository.", "body_md": "Put two people on one repo. One writes code in a web chat: Claude.ai, ChatGPT, whichever tab is open. The other lives in the terminal and the editor. Same repo, same pull request, two completely different loops.\n\nThe feature arrives looking finished. Clean functions, sensible names, a comment that almost reads like it was written by a person who had to maintain this. Then the terminal side pulls. Compose does not boot. A port is already taken. A migration assumes a column the last branch never added. The unit tests the chat invented were never run against this tree. Someone else is now the person who finds that out.\n\nThat is not a talent gap. It is an interface gap. And it has a boring name once you stop calling it \"the other person uses ChatGPT.\"\n\nThe easy version of this story is web apps versus IDEs. It is also the wrong version.\n\nClaude Code now runs in a browser tab. Cloud agents spin up a VM and look, from the chat, like they \"ran the tests.\" A Codespaces session is a website with a real shell behind it. Meanwhile an IDE with the terminal tool turned off is just a prettier paste buffer.\n\nThe split is whether the model is inside a **runtime loop** that can touch the project you actually ship:\n\nA document editor can do step 2 if you copy hard enough. It cannot do 1, 3, or 4 against *this* machine, *this* compose file, *this* `.env` that is not in git. The model predicts what might work. You find out later.\n\n|  | Web chat (a document) | A harness (a runtime) | \n|---|---|---|\n| What the model sees | Whatever you pasted | The tree, the shell, the failing test | \n| What it emits | Text about a system | Edits inside the system | \n| Stop condition | The reply looks done | A command exits 0 | \n| Who pays for retry | The person whose checkout has to boot | The model, against compiler output | \n\n```\nweb chat   prompt → text → paste → git → (someone else) → tests\nharness    prompt → edit → test → fail → edit → test → git\n```\n\nWeb chat is excellent at architecture, rubber-ducking, and first drafts of docs. I still use it for those. The moment the output is supposed to land in a shared repository, the chat is generating text about a system it cannot see.\n\nVendors already have a word for the loop. I did not invent it, and I am not going to pretend the word is obscure. [LangChain's line](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) is the cleanest: **agent = model + harness.** If you are not the model, you are the harness. A raw model cannot keep state, execute code, or install a package. The wrap around it can. Anthropic describes the same loop as gather context, take action, verify results, and Claude Code's own docs tell you to ask it to write the tests, run them, and fix the failures. [Aider](https://aider.chat/docs/usage/lint-test.html) does it with fewer adjectives: `--auto-test` runs your suite after each edit, feeds a non-zero exit back, and tries again.\n\n[Böckeler, on Fowler's site](https://martinfowler.com/articles/harness-engineering.html), splits the wrap into guides (steer before it acts) and sensors (observe after, so it can self-correct). Tests and linters are the cheap sensors. Web chat, pointed at this repo, has neither. Paste-back of a screenshot is a sensor you operate by hand, once, badly.\n\nThis is not a branding fight. Same model, different wrap, different score. LangChain reported moving a coding agent from outside the top 30 to top 5 on Terminal-Bench 2.0 **by changing only the harness**. Aidan Li's write-up of CORE-Bench is the same shape: Claude Opus 4.5 at 42% on the standard CORE-Agent scaffold, 78% on the Claude Code scaffold. The model did not get smarter between those two numbers. The loop around it did.\n\nThe website answers. The harness operates. I want to be precise, because the word is doing a lot of work in public. Here it means tools, a workspace, a shell, and a stop condition that is not \"the reply looks good.\" It does not mean a product, and it does not mean the chat UI grew a dark theme.\n\nIf you have used a CLI or an agent mode that actually shells out, you already know the feeling. The first response is still wrong half the time. The difference is who pays for the second through fifth attempts. In a harness, the model does, against compiler output. In a web chat, a human does, usually the one who did not write the patch.\n\nWhen those two loops dump into one repository, speed becomes a transfer, not a gain.\n\nThe web-chat side feels fast because the expensive part never ran. No bind-mount, no schema that only exists after migrate, no \"this test needs Redis and Redis is not in the prompt.\" What ships is a plausible diff.\n\nThe IDE or CLI side then does a second job that is not on the sprint board: make the plausible diff inhabit the real environment. Ports, env vars, the test that was asserting a happy path the chat assumed, the merge that only conflicts because two prompts invented the same helper. That is the context tax. It is paid in the checkout that has to boot, not in the tab that wrote the patch.\n\nI do not think this makes the web-chat person careless. Paste-back of an error is a harness, just a terrible one. You screenshot a traceback, the model apologizes, you paste a new file. Each round strips the repo. The model never saw `docker compose ps`. It never saw that CI uses a different Python than the snippet. It is vibe coding with a colleague as the runtime.\n\nCI does catch some of this. It catches it late, on a shared queue, in a log the author may not read. CI is the team's last harness. It is a bad first one. If you already know who the cleanup crew is, name it. \"They ship so fast\" usually means the bill arrives in someone else's working tree.\n\nIt is not \"ban the browser.\" A cloud coding surface with a real shell is closer to my terminal than it is to a chat transcript. If someone wants to stay in a tab, the move is Codespaces, or Claude Code on the web, or whatever still runs `pytest` against the project. The window can stay. The loop cannot leave.\n\nIt is not \"IDE agents are correct.\" They ship broken code constantly. They just fail in a place you can see before `git push`. The METR result that people felt faster than they were measured is a Cursor study, not a ChatGPT-paste study, so I will not pretend it proves this tax. It only proves that felt speed is a rotten metric.\n\nIt is not a reason to skip review. A green local run is not understanding. Willison's line still holds: if you could not explain the diff, you are not done. The harness closes the cheap failures. The expensive ones remain human.\n\nAnd it is not a mandate to standardize on one vendor. Shared `CLAUDE.md` / agent rules / the actual test command in the README: those travel. The specific app does not have to.\n\nThe tax exists because the team does not share a runtime. Two people, two loops, one git remote. Git is not a runtime. It is a merge queue for text.\n\nThat shared runtime has a name, and it is the same one as above: a **harness** the whole team can fail in. Same tree. Same commands this repo uses to decide if a change worked. A stop condition the author of the patch can see. Codespaces. A project VM. Claude Code or Aider or Cursor pointed at this checkout. A human rule that no PR opens until the suite ran *here*. Fowler's cheap sensors, in the loop, before merge. The window can differ. The harness cannot be optional for the person who writes production code.\n\nThat may be enough for the two-loop problem. It does not require a new stack. It requires that production code is not authored in a document the repository cannot execute.\n\nWhen the work itself is more than one process, the same hole shows up one layer up. A tool server in one chat. Someone else's agent in another. A graph a teammate deployed last month. Each piece can look finished in its own tab, inside its own little harness. Nothing has verified the run they were supposed to be part of. Sharing Cursor does not close that. A harness that only wraps one repo will not either. One place the composed work has to pass might.\n\nI still want the web chat for the parts that are actually text. Specs, scary design questions, \"is this even a good idea.\" The moment we are editing the system, I want a harness the whole team can fail in, not a cleanup crew pretending to be one.\n\nIf your team shared one harness tomorrow, would the web-chat patch still land, or would it die at the door? And if the work already crosses more than one process: is git still the only thing you share?\n\n**Sources and further reading:**\n\n`--auto-test` / `/run`.", "url": "https://wpnews.pro/news/web-chat-is-not-a-coding-environment", "canonical_source": "https://dev.to/azank1/web-chat-is-not-a-coding-environment-2m77", "published_at": "2026-09-23 10:14:50+00:00", "updated_at": "2026-09-23 10:29:02.121397+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "large-language-models", "ai-products"], "entities": ["Claude.ai", "ChatGPT", "Claude Code", "LangChain", "Aider", "Terminal-Bench 2.0", "CORE-Bench", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/web-chat-is-not-a-coding-environment", "markdown": "https://wpnews.pro/news/web-chat-is-not-a-coding-environment.md", "text": "https://wpnews.pro/news/web-chat-is-not-a-coding-environment.txt", "jsonld": "https://wpnews.pro/news/web-chat-is-not-a-coding-environment.jsonld"}}