# AI Made Prototypes Free. Production Didn't Get Cheaper. > Source: > Published: 2026-08-01 13:08:12+00:00 [AI](https://sourcefeed.dev/c/ai)Article # AI Made Prototypes Free. Production Didn't Get Cheaper. New data from Veracode, Stack Overflow, and METR keeps confirming that the demo was never the hard part. [Mariana Souza](https://sourcefeed.dev/u/mariana_souza) Every engineering team now has a version of the same story. Someone spins up a working demo in an afternoon — an internal dashboard, a customer-facing feature, a whole product skeleton — and the room lights up. Then three months later the thing still isn't in production, and everyone's quietly wondering what went wrong. Anuradha Weeraman's essay "[The prototype isn't the product](https://weeraman.com/the-prototype-isnt-the-product/)," which made the rounds on Hacker News this week, names the misconception underneath: AI collapsed the feedback loop for the *first* working version, and people mistook that for collapsing the distance to a *shippable* one. Weeraman is right, and it's worth spelling out why — because the argument is older than it looks, and the data backing it up is newer than most teams' planning assumptions. ## We've made demos cheap before The prototype-to-production gap isn't an AI discovery. Tom Cargill's ninety-ninety rule — the first 90% of the code takes 90% of the time, and the remaining 10% takes the other 90% — dates to the 1980s. Fred Brooks made the deeper cut in "No Silver Bullet": tools can eliminate *accidental* complexity, the drudgery of expressing a solution, but not *essential* complexity, the irreducible difficulty of specifying what a system should do under every condition that matters. Every generation since has produced a technology that made the demo cheap. Fourth-generation languages. Visual Basic. Rails scaffolding, which famously produced a blog engine in fifteen minutes. Low-code platforms. Each one genuinely compressed the early loop, and each one produced a wave of organizations that confused demo velocity with delivery velocity, then rediscovered — expensively — that error handling, data modeling, security, and scale were still waiting at the end. LLM coding tools are the most powerful entry in this lineage by a wide margin, and that's exactly why the confusion is worse this time. When the accidental complexity drops to near zero, the essential complexity doesn't shrink — it just becomes the *entire* remaining bill, and it arrives with no line items. ## The bill, itemized The numbers landing over the past year put shapes on that bill. [Veracode](https://www.veracode.com/resources/analyst-reports/2025-genai-code-security-report/)'s 2025 GenAI Code Security Report ran 80 curated coding tasks across more than 100 LLMs and found the models introduced known vulnerability classes in 45% of cases — with Java failing around 70% of the time and cross-site scripting defenses missing in 86% of relevant samples. The finding that should worry you most: newer, bigger models wrote more *functional* code but no more *secure* code. Security performance stayed flat as capability climbed. The gap isn't closing on its own. Meanwhile [Stack Overflow](https://survey.stackoverflow.co/2025/)'s 2025 developer survey found 84% of developers using or planning to use AI tools while distrust of their output climbed to 46%, up from 31% the year before. The top frustration, cited by 45% of respondents, was "AI solutions that are almost right, but not quite" — and 66% said they're spending *more* time fixing almost-right code. That's the prototype-production gap measured from the inside: the code runs, the demo works, and the correctness debt is invisible until someone pays it down. Even the productivity headline deserves a caveat in both directions. [METR](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)'s randomized trial found experienced open-source developers completed real tasks 19% *slower* with early-2025 AI tools — while believing they'd been 20% faster. METR's 2026 follow-up cohort, with newer tools and more tasks, softened that to roughly a wash. Take the honest reading: on mature codebases with high quality bars — production work, in other words — measured AI speedups range from negative to modest, and developer self-perception is unreliable. The 10x gains are real, but they live at the prototype end of the pipeline. ## What to actually do about it The practical shift is treating AI output the way you'd treat a pull request from a fast, confident stranger: useful, high-volume, and untrusted by default. Concretely, that means the gates you may have let slide become load-bearing. SAST and dependency scanning in CI stop being compliance theater when 45% of incoming code plausibly carries a flaw the author — human or model — never considered. Review capacity becomes your bottleneck, so protect it: smaller diffs, generated tests reviewed as skeptically as generated implementation, and a hard norm that nobody merges code they can't explain. The organizational failure mode is subtler and more expensive. Call it the demo-driven roadmap: a stakeholder watches a v0 or Lovable prototype come together in an hour and recalibrates every estimate downward accordingly. The correct response isn't to hide the prototypes — they're genuinely valuable for killing bad ideas cheaply. It's to re-plan around the new cost structure: exploration is now nearly free, so do much more of it, but hardening costs what it always cost, so budget it explicitly instead of assuming the demo was 80% of the work. It was 20%, same as ever. And the hiring implication cuts the way Weeraman says it does, uncomfortably. The engineer whose value was typing correct syntax quickly is competing with a $20-a-month subscription. The engineer who can look at generated code and see the full table scan, the missing idempotency key, the auth check that only covers the happy path — that person's leverage just multiplied, because they can now review and harden output at a volume no 2020 team could produce. ## The gap is the job now Here's the judgment call: this is a genuine shift, not hype — but the shift is in *where the work lives*, not in how much of it exists. AI ate the part of software engineering that was always cheapest per unit of business value and left the expensive part intact, which means production engineering skills didn't get commoditized. They got scarcer relative to demand, because there's suddenly ten times more prototype-grade code applying for promotion to production. Teams that internalize this will use AI to explore ruthlessly and ship deliberately. Teams that don't will keep discovering, one incident at a time, that the prototype was never the product. ## Sources & further reading - [AI doesn't generate working products, that's still your job](https://weeraman.com/the-prototype-isnt-the-product/)— weeraman.com - [2025 GenAI Code Security Report](https://www.veracode.com/resources/analyst-reports/2025-genai-code-security-report/)— veracode.com - [2025 Stack Overflow Developer Survey](https://survey.stackoverflow.co/2025/)— survey.stackoverflow.co - [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)— metr.org [Mariana Souza](https://sourcefeed.dev/u/mariana_souza)· Senior Editor Mariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon. ## Discussion 0 No comments yet Be the first to weigh in.