{"slug": "what-a-second-model-cannot-catch", "title": "What a second model cannot catch", "summary": "A developer argues that while AI can automate much of software engineering, the irreplaceable human contribution lies in intent and judgment, not verification. The developer describes how AI agents caught each other's errors, but only a human could distinguish intended behavior from defects and make product decisions.", "body_md": "A lawyer told me his job was safe, and gave a better reason than most\n\npeople give.\n\nHe catches AI mistakes constantly, he said, but that is not the point. The\n\npoint is where his work lives. Roughly 20% of it happens in courtrooms and\n\nin private negotiation — rooms that produce no transcript anyone will ever\n\ntrain on. The rest, the drafting and the research, is already in the\n\ncorpus. His conclusion: \"AI at best can rise to the level of what a law\n\nstudent knows after graduating.\"\n\nI answered honestly, and the answer was worse for me than for him:\n\nYour 20% is protected because it never gets written down. My work is\n\n100% digital — all the code, all the solutions, everything is basically\n\nthe training data. Maybe my whole job is that 80%.\n\nThat is the uncomfortable version of the question. Not \"will AI replace\n\ndevelopers\", which is exhausted, but: **if none of my work is protected by\nscarcity, what is left that is actually mine?**\n\nThe obvious answer — orchestration, verification, taste, being the\n\ninterface between the human and the machine — is also the answer everyone\n\ngives, which is a good reason to distrust it. So I spent a stretch of work\n\nwatching for it specifically. Not asking whether steering matters. Asking\n\nwhat it catches that nothing else does.\n\nThe result splits cleanly in two, and the split is more useful than the\n\nslogan.\n\nA large share of the \"only a human can\" list is not human work at all. It\n\nis verification work, and a second model does it well — often better than\n\nthe first one did, because it has no stake in the first answer being\n\nright.\n\nI don't have to be in the room for this part, and mostly I'm not. Over one\n\ntwo-day stretch my agents caught each other repeatedly: a reviewer refuted\n\na diagnosis another agent had already written up, an executor stopped\n\ndead on a brief whose premise was wrong and explained why, a fresh pass\n\nturned up ten defects in code a previous pass had promised to check and\n\nquietly skipped — including one that charges a user a fee and then denies\n\nthe request.\n\nNot one of those needed me. My contribution was structural, not\n\nintellectual: a setup where a second opinion is cheap, and rules that make\n\nrefusal legal — an agent that hits a false premise is allowed to stop\n\nrather than build on it.\n\nThat is real, and it is not nothing. But it is also **automatable**, and\n\nanyone reading this can copy it in an afternoon. If this were the whole\n\nanswer, the lawyer would be right about me.\n\nThen there is the other category, and it behaves differently.\n\n**Case one.** A feature had been built, verified, and shipped. Six\n\nindependent verification passes came back green. Every one of them\n\nconfirmed the code did what the spec said, and none of them was sloppy —\n\nthe spec wasn't wrong either.\n\nThen I used the product, and the feature was silent. A single line on the\n\nclient discarded the value the server had just computed, so the whole\n\nthing ran end to end and produced nothing a person could hear.\n\nThe lesson isn't \"human testing matters\", which everyone already agrees\n\nwith and nobody schedules. It's narrower and less comfortable: **every one\nof those passes was asking whether the code matched the description.** Not\n\n**Case two.** My agents handed me a list of nine defects queued for\n\nfixing. I removed three of them on sight — not because the analysis was\n\nwrong, but because the behaviour was **intended**. A toast notification\n\ncovering the navigation bar is a defect if you assume it shouldn't; it's a\n\ndesign decision if I decided it should. No verification pass can separate\n\nthose two, because the difference isn't in the code. It's in what somebody\n\nwanted, and I'm the one who wanted it.\n\n**Case three.** The plan on my desk was to build a new empty-state\n\ncomponent for a slot that was stuck showing an infinite loading animation.\n\nI stopped it: the component already had a paused state, because I had\n\nasked for one when the design system was built. The fix went from a new\n\ncomponent to a single boolean.\n\nThere is a pattern, and it is not \"humans are wiser\". In all three cases\n\nthe machine had every fact and still could not get there, because the\n\nmissing input was **not a fact**. It was intent — and intent lives in the\n\nperson who formed it, not in the artifact.\n\nA verifier can prove a system does what it says. It cannot tell you\n\nwhether what it says is what you meant. That gap does not close with a\n\nbetter model, because it isn't a knowledge gap.\n\nAny version of this argument that only lists wins is an advertisement, so\n\nhere is the stretch where I lost — and the interesting part is exactly\n\n*where* I lost it.\n\nI'm a frontend developer. The work in question was backend: a caching\n\nlayer I can read but could not have designed, in a language I don't write\n\ndaily. And that is precisely where I loosened up. Not on how much work I\n\nhanded over — on how much **judgement**. In my own domain I argue with the\n\nmachine constantly, because I can feel when an answer is shaped wrong.\n\nOutside it I have no such feel, so I let conclusions through.\n\nThe conclusion that got through: a cache would serve stale data. It was a\n\ngood-looking claim. The cache was real, its key was genuinely missing the\n\nfield it was said to be missing — and the whole thing sat on a code path\n\nthe application never takes. Ten call sites, not one of them reaching it.\n\nA single search settles it. For two days nobody ran the search, and I\n\nwasn't equipped to know one was needed.\n\nThe cost wasn't a bug. It was a plan: work split across two repositories,\n\none of them another team's, and a dependency communicated that did not\n\nexist.\n\nIt did get caught, and how it got caught is the part worth keeping. Not by\n\nme spotting the flaw — I could not have. By me refusing to sign off. I\n\nkept pushing the plan back and forth, asking for it to be exact, making\n\nthe pieces line up, and the contradiction eventually surfaced under that\n\npressure. All of the errors came out in the end, but not one of them came\n\nout because I knew better.\n\nSo the thing the seat supplies isn't expertise — I had none here, and it\n\nstill worked. It's the refusal to accept \"close enough\" from a system that\n\nis fluent, fast, and confident. That is a much smaller qualification than\n\ndomain knowledge, and much easier to give up, because giving it up costs\n\nnothing today and feels like trust.\n\nBack to the lawyer. His 20% is protected by scarcity: those rooms produce\n\nno data. That protection is real but passive, and it only holds while the\n\nrooms stay closed.\n\nThe seat I'm describing has a different property. It isn't protected by\n\nmissing data — it's the seat that **supplies** something data cannot\n\ncontain, which is what \"correct\" was supposed to mean in the first place.\n\nNo corpus confers that, because it was never a fact about the world; it\n\nwas a decision somebody made.\n\nBut I want to be precise about the size, because this is where these\n\narguments usually inflate. That seat is not 80% of my job. Most of my week\n\nis still specification, review, and repair — all of it in the corpus, all\n\nof it increasingly done better and faster by the machine. The part that is\n\nstructurally mine is small, intermittent, and mostly consists of knowing\n\nwhat I actually wanted.\n\nWhich leaves the real risk, and it isn't replacement.\n\nMy wife doesn't work in tech. She read an earlier version of this and gave\n\nme the sharper version of it:\n\nI changed my mind a bit about AI and robots. It's not going to be some\n\nuprising because they don't need us. Humanity will be at fault, because\n\nit'll stop controlling and checking, and it'll just flow that way on its\n\nown.\n\nAnd she named the cause without softening it:\n\nBecause of human stupidity and degradation, in a way. Like — whatever,\n\nwhy check, it does it itself.\n\nWhat happened next is the reason this section exists at all. I put that to\n\none of my agents, and it returned a clean economic account: checking is\n\nboring and expensive, the machine is right most of the time, the cost of\n\nverifying exceeds the expected value, so people rationally stop. Tidy,\n\nplausible, and wrong in a way that is hard to see, because it is the kind\n\nof wrong that reads as insight.\n\nThere is no economic argument. \"I don't care\" is not a weighed cost — the\n\nweighing never happens. That is exactly what makes it durable: you cannot\n\nrefute a decision that was never made. Give it a model and you have\n\nalready mistranslated it into something answerable.\n\nThe operational version isn't \"checking is too expensive\". I can write it\n\nout myself, because it is already how I think on a good week, when\n\neverything is landing:\n\nI just won't check, because it all works anyway. Why bother doing extra\n\nwork if everything's fine. When the mechanism visibly starts failing,\n\nthen sure, you can check. But by then it's already too late.\n\nSo the seat doesn't disappear because something takes it. It disappears\n\nbecause sitting in it stops feeling necessary — and the signal that it was\n\nnecessary arrives after the fact, if it arrives at all.\n\nHer conclusion was one sentence, and I have not improved on it: you can't\n\nlet it run by itself, you have to keep some control and know what's going\n\non.\n\n*The lawyer exchange is from a Reddit thread. The framing about training\ndata, and the closing line — that \"mashing keys\" disappears while \"being\nthe interface between the human and the machine\" remains — are his.*", "url": "https://wpnews.pro/news/what-a-second-model-cannot-catch", "canonical_source": "https://dev.to/dimonb19a/what-a-second-model-cannot-catch-480c", "published_at": "2026-08-23 13:16:09+00:00", "updated_at": "2026-08-23 13:43:39.989397+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/what-a-second-model-cannot-catch", "markdown": "https://wpnews.pro/news/what-a-second-model-cannot-catch.md", "text": "https://wpnews.pro/news/what-a-second-model-cannot-catch.txt", "jsonld": "https://wpnews.pro/news/what-a-second-model-cannot-catch.jsonld"}}