# Even Anthropic didn't notice Claude got worse for weeks — AI quality is invisible, and that's the enterprise problem > Source: > Published: 2026-06-16 03:04:12+00:00 The company that ships the best coding model on the planet just published a postmortem worth sitting with: three innocent-looking config changes quietly degraded Claude's output — and it took **weeks** to track down. If the team that knows AI best can fail to notice their own model getting worse, what makes a business think it can eyeball an agent's output and catch the rot? Speed isn't in question. *Whether the quality holds — and whether you'd even notice if it didn't* — is the whole game. ## Two signals: speed is settled, "is it still good?" is not - **Anthropic's own postmortem.** Their internal investigation confirmed that three independent changes in March–April 2026 — lowering Claude Code's default reasoning effort, a cache bug that wiped session data every turn, and a system-prompt revision aimed at reducing verbosity — *collectively* degraded output quality. Note: not model rot — **config-layer drift**, with **no single alarm**, found only weeks later via user feel and investigation. - **Canva's CTO** said plainly that "vibe coding" isn't fit to ship straight to production; core systems need the full loop of *AI-generate → human dehydrate/restructure → test coverage → security scan*. The real-world CTO failure log for "ship the AI output directly" includes DB query crashes, permission holes, broken auth flows, and off-by-one logic bugs. Put together: **speed, AI proved long ago. What's unsolved is that AI output quality drifts — silently — and you often don't know it drifted.** ## Why "you won't notice" is the real enterprise problem The scary part of the Anthropic case isn't "a bug happened." It's that it stayed **invisible for weeks.** Each of the three changes looked reasonable alone; together they stepped quality down — with no single point of failure to page anyone. Scale that to enterprise apps and it amplifies: - An agent opens dozens of PRs a day; every one "looks right." - Changes scatter across permissions, validation, reconciliation, approvals — each plausible in isolation. - Nobody can verify "did this batch actually get worse?" one by one. - By the time the business breaks (wrong totals, privilege escalation, a skipped approval), weeks have passed. **This is the Anthropic postmortem, structurally — just amplified.** You can't "look harder" your way to AI quality, because regressions are often silent, cumulative, and spread across many spots. Anthropic of all teams got caught by exactly that. ## Welding the quality floor into the architecture This is the core idea behind **Oinone** — AI-native, but with rigor that doesn't depend on *noticing*; it lives in the architecture: - **The AI emits metadata, not code.** "Add a 3-level approval to the quote object" produces a structured metadata diff of model/view/flow/permission — a few dozen readable lines, not a wall of code you can only "trust by vibe." Quality is *scannable by eye*, not inferred from feel. - **The quality floor is enforced by the framework, not by the AI's diligence.** Permission model, data validation, transactional consistency, audit — the "drift here = serious incident" parts — are framework-enforced. The AI can't move them and can't route around them. It *can't silently get worse* there, because it never touches those red lines. - **Every change is diffable, rollbackable, traceable.** Anthropic spent weeks localizing three changes; with metadata, each change is a structured diff — wrong, roll the whole thing back; what changed is obvious. "Needle in a haystack, after the fact" becomes "visible at commit time." - **Change once, consistent everywhere.** A model change derives UI/API/permissions in sync — no "changed the field, forgot the permission," the most classic silent regression, and exactly the kind of "multi-spot, no single alarm" drift the Anthropic case is made of. One line: **Speed by AI, rigor by Oinone.** AI quality drifts and degrades silently — that's its nature, Anthropic included. Oinone doesn't bet on *you catching it*; it welds the quality floor into the foundation so the output simply can't drift into the danger zone. ## Three questions for anyone evaluating tools - **How do you know this batch of AI output didn't quietly get worse?** Human feel and after-the-fact investigation (Anthropic took weeks), or a structured diff that makes regression visible at commit time? - **Who backstops the quality floor?** Hoping the AI and devs "remember to do it right," or a framework layer the AI can't even reach? - **Would you hand a core system to an AI that can silently degrade?** A wall-of-code system lets you wait for the incident; a metadata-driven, framework-backstopped one keeps the high-risk zone out of the AI's drift range entirely. ## FAQ **Q: What actually happened at Anthropic?** A: Per their postmortem, three independent March–April 2026 changes (lower default reasoning effort in Claude Code, a cache bug wiping session data each turn, a system-prompt trim) stacked up and quietly lowered output quality — found only weeks later. AI quality regressions can be silent, cumulative, and spread across many places. **Q: What's this got to do with low-code / Oinone?** A: Building apps with AI hits the amplified version of the same problem — lots of agent output, all "looks right," silent drift. Oinone makes the AI emit architecture-constrained metadata, with the quality floor (permissions/validation/consistency) enforced by the framework — not dependent on "noticing in time." **Q: Is it open source?** A: Yes (AGPL-3.0). One `docker compose` and it's up in ~5 minutes; self-hosted, data never leaves your environment. It runs in the core systems of billion-scale enterprises. If this framing helped, the project is open source (AGPL-3.0) — a ⭐ supports the maintainers: (Disclosure: I work with Oinone.)