cd /news/developer-tools/why-using-ai-to-catch-ai-drift-is-th… · home topics developer-tools article
[ARTICLE · art-54764] src=dev.to ↗ pub= topic=developer-tools verified=true sentiment=· neutral

Why "using AI to catch AI drift" is the wrong thing to do

A developer argues that using AI to catch AI-generated code drift is the wrong approach, citing non-deterministic results, compounding costs, and opacity. Instead, they advocate for deterministic rule engines like ReWeaver, which provide consistent, inspectable checks between design source and code.

read5 min views1 publishedJul 10, 2026

Something's been bugging me about how most teams are handling AI-generated UI code right now.

The pattern goes: Cursor, Copilot, v0, or Claude Code writes the component. It looks right. It compiles. It probably even matches the Figma file at a glance. Then six sprints later someone notices the button padding doesn't match the design tokens anymore, or the state was never implemented, or there's a hardcoded hex value sitting in a component that's supposed to pull from the theme. Nobody did that on purpose. It just... drifted.

The numbers back this up if you don't already feel it in your own codebase. Faros AI's 2026 engineering report found code churn is up 861% against pre-AI baselines, and 31.3% more PRs are now merging without any human review at all. Humans didn't get worse at reviewing. There's just more to review than any team can keep up with, and a lot of what's slipping through isn't broken code — it's code that's quietly disagreeing with the design system it's supposed to be implementing.

So teams reach for the obvious fix: point another AI model at the diff and ask it to catch what the first one missed.

I think that's the wrong architecture, and it's worth explaining why.

Using an LLM to catch drift introduced by another LLM has three structural problems that don't show up until you're relying on it in production: It's non-deterministic. Ask the same model to review the same diff twice and you can get two different answers. That's fine for a suggestion. It's not fine for something you want to gate a merge on, because "sometimes it catches this class of bug" isn't a policy, it's a coin flip.

It's expensive in a way that compounds. Every scan is more tokens on top of the tokens that generated the code in the first place. At CI scale, across every PR, that adds up fast — and it's a cost that scales with your team's AI usage instead of shrinking it.

It's a black box checking a black box. If a model flags something, can you explain why to a teammate, in a way that's the same explanation every time? If not, you've just added a second layer of "trust me" on top of the first one.

None of this means detection should be AI-powered. It means the checker needs a different design than the thing it's checking.

A deterministic engine — same input, same output, every single time — isn't just a nice property. It's what makes a check trustworthy enough to gate a merge on in the first place. If a rule flags something, it flags it because a specific, inspectable condition was true, not because a model's confidence crossed some threshold on a given run. That means:

This is the architecture we went with for ReWeaver: a versioned rule engine that runs the same checks over your design source and your code, surfaces drift as a diff, and never asks you to trust a probability score.

"Design-code drift" gets used loosely, so it's worth being specific about what it actually covers. We ended up mapping it across nine dimensions, because "does the code still match the design" turns out to be a much bigger question than color and spacing:

A hardcoded color and a missing error boundary are both "drift" in the sense that matters here: the code no longer agrees with the intent it was supposed to implement, and nothing forced a human to make that decision on purpose.

A checker that only flags issues just moves the bottleneck. You still need someone to triage every finding, which doesn't scale any better than manual review did.

So, findings come with a proposed fix, not just a flag. And when something genuinely should be suppressed — a false positive, a deliberate exception — that suppression has to be an explicit, git-visible decision (a //reweaver-ignore

with a reason), not a silently ignored warning. Given that nearly a third of PRs are merging without review at all, a decision gate only does anything if someone is actually the one making the decision. The goal isn't to add another layer of noise on top of an already-overwhelmed review process — it's to make sure the decisions that do get made are visible and on the record.

It's also model-agnostic by design. It doesn't matter whether the code came from Cursor, Copilot, Claude Code, or v0 — the same rule engine runs regardless, and it doesn't need extra API tokens to do it. It runs inline in VS Code, does a round-trip sync with Figma (not just one-way detection), and runs as a GitHub Action on every PR — no new dashboard, no workflow change.

We're in private beta and looking for a small number of teams actually shipping AI-generated frontend code to run this against a real repo and tell us what you find. Even if it's just one small workflow, your feedback is valuable.

The beta is developer-led, not a sales process — install it, point it at something real, see what surfaces. You get direct access to the founder and engineering team for bugs, questions, and feedback while we build this out.

(Current caveat, in the interest of not overselling: it runs primarily on Mac today. Windows support exists in beta but has some rough edges we're still smoothing out.)

If you want to see the detection engine before committing to anything, there's a no-signup Playground where you can paste a snippet of real AI-generated code and get a live drift score. If you're interested in the full beta — running across your actual codebase, Figma files, and PRs, that's here. Either way, if you're dealing with this problem, I'd genuinely like to hear how you're handling it today, even if it's not with us.

── more in #developer-tools 4 stories · sorted by recency
── more on @cursor 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/why-using-ai-to-catc…] indexed:0 read:5min 2026-07-10 ·