AI AI slop.
We know it when we see it, but it’s tough to actually define what makes certain code slop.
The problem is that we use AI slop to describe a bunch of different failures. They don’t have the same cause, and they don’t need the same fix.
So here, I’m going to look at which patterns we can describe well enough to lint, what needs deeper architectural work, and how to tell whether any of that cleanup actually makes your software better. Along the way, I’ll use examples from our work on Agent-Native—an open-source framework we’ve been making at Builder where humans and AI agents operate the same product through shared actions, state, and permissions—including a local experiment with anti-slop lint rules and agents trying to satisfy them.
What are we actually calling AI slop? #
Think about what makes you hesitate when you review a change.
Sometimes a value has already been validated, but an agent widens it to unknown, passes it through three layers, and asserts it back into the type it started with.
Sometimes two helpers do almost the same thing, except neither quite fits the next task, so someone writes a third. Or the code fills up with defensive checks for inputs the system can’t actually receive.
Then there’s code that looks tidy, typechecks, passes lint, and still falls apart when someone tries the real user flow.
A needless assertion might take a few lines to remove. Almost-duplicate helpers might require a decision about where a responsibility belongs. A broken flow requires reproducing what happens when the pieces run together.
Before asking an agent to “clean this up,” pick a behavior or recurring problem you can describe. “The import flow keeps losing the validated type” gives you somewhere to start. “This repository feels sloppy” leaves the agent guessing which changes you’ll like.
How missing context creates more mess #
One reason these patterns accumulate is that the information needed to make a good change lives somewhere else.
A value was validated three files ago, but the agent working here hasn’t found that code. Or a helper already handles this case, but its name and location don’t suggest that. The agent adds another check, fallback, or helper to finish the task with the information it has.
Humans do this too. Agents can produce a lot of these decisions quickly, and the next agent may have to discover the same facts all over again.
This is the locality problem: code that changes together should be easy to find together. If understanding one behavior requires a tour through seven directories, the repository makes every change harder before anyone has written a line.
When an agent has to touch a route handler in api/, find a validation schema in shared/, trace a helper in utils/transforms/, and inspect a database model in server/, it runs out of working context and defaults to synthesizing a local fallback check or casting to unknown.
So, keep related decisions easy to follow, even when they live in separate files: where input becomes trustworthy, which module owns the behavior, and what callers can rely on.
You can improve some of that with a small local repair. Other cases need architectural work. Start by gathering enough evidence to tell them apart.
Run a few rules, then inspect what they found #
Dillon Mulroy’s anti-slop provides opinionated Oxlint rules for patterns such as chained assertions, broad internal types, and assertions without a nearby explanation. They give you specific places to investigate.
For example, this deliberately simplified code validates a value, throws away its type, and then asserts the type back:
If the schema already returns an `Account`, you can keep that information:
The code gets shorter because the detour served no purpose. You don’t need a repository-wide refactor to fix it.
But a rule sees only what its analysis can establish. These rules generally inspect the code in front of them; they don’t reconstruct every value’s history across the whole repository. A broad type at an external input boundary can be appropriate. A runtime check may be the thing that makes an input safe.
Decide which findings deserve a change
When we ran the rules across a pinned revision of Agent Native, they produced 41,280 warnings. Reviewing a sample turned up local repairs, architectural questions, duplicate findings, and correct code we should leave alone. The count itself didn’t tell us which was which.
Experiment note: the local scan and repair trials didn’t adopt rules or apply patches to Agent Native. The examples here describe findings and trial outcomes.
Three examples show how differently a warning can turn out:
- A local repair: a Zoom token response was asserted into the expected type without validation establishing that shape.
- An architectural question: a Clips import test mocked several modules we own, including security, persistence, and uploads. We needed to examine what the test could actually prove before deciding which mocks to replace.
- No change needed: a conditional object spread added a field only when a validated value existed. Omitting the field was the intended behavior.
Before enabling a rule as an error, inspect a few findings in the kinds of files where it will run. Follow the value back to its source. Read the caller and the relevant tests. Decide what a correct repair would look like, including when the existing code is already correct.
Ask an agent to collect that context before editing. A small report with the finding, the surrounding contract, and a proposed action is easier to review than a hundred “cleanup” changes.
How to introduce the rules without wrecking your codebase #
Once you’ve found a rule that catches a recurring problem, introduce it in a scope you can review.
Run it without automatic fixes first. Pick one package, one feature, or one recurring pattern. Keep the output as a baseline so you can tell whether your next change introduced a new problem or merely encountered an old one.
If the existing findings are numerous, decide how to handle them before making the rule block everyone’s work. Depending on your tooling, you might enforce it in a smaller directory, track existing findings separately, or repair a manageable group before enabling it more broadly. Be careful with a policy that checks only changed files. It can make a tiny unrelated edit responsible for years of accumulated warnings. The team should know whether the rule asks them to fix a new violation, clean the whole file, or leave a documented exception.
Keep cleanup batches small enough that someone can explain the changes. Mixing type repairs, module moves, test rewrites, and formatting into one diff makes it harder to tell which change broke a behavior.
And if a rule keeps objecting to correct code, adjust its scope or remove it. A collection of suppressions can tell you that the rule doesn’t fit the system.
Tell the agent what the repair must accomplish #
A lint warning makes a very convenient target. An agent can optimize for making it disappear, even when the code still has the problem that prompted the rule.
We saw this in an Analytics trial. An agent replaced a broad object parameter with a more specific type derived from the resolver. The warning disappeared, typechecking passed, and all 23 tests in the focused suite passed. The code still used unvalidated input.
Another attempt, prompted to establish the validation contract, introduced a named Zod schema and satisfied that requirement. Both attempts got green checks.
A Chrome message trial exposed the same failure: one change removed the flagged typeof pattern and passed the checks while still dispatching the incoming message without parsing it. The attempt that did validate the message introduced internal any, so it still needed review.
These were a few controlled examples, not a measure of how often agents make this mistake. They’re enough to make me wary of using “the warning is gone” as the acceptance condition.
Write the acceptance condition first
When you ask for a repair, describe what the program must know or do afterward. For a message handler, that might look like this:
Validate the incoming message before dispatch. The handler should receive a named command type whose required fields have been checked. Preserve the existing behavior for valid messages, handle invalid messages explicitly, and remove the warning without adding an unchecked assertion or widening the value to any.
You can inspect where validation happens and test whether malformed input reaches the handler.
The right contract depends on the system. You might need to establish that an authorization check happens before a write, that a failed upload doesn’t leave a completed record, or that a value stays typed after parsing. Name that requirement before the agent starts editing.
If the agent removes a warning by adding a comment, moving a cast, or changing the spelling of a check, ask what new guarantee the program gained. Sometimes a comment documents a real guarantee that already exists. Sometimes it just documents an assumption nobody has checked.
Refactor when the same design keeps producing the mess #
If five callers all repeat the same defensive checks, fixing each caller may leave the cause intact. Follow the data far enough to understand why every caller thinks it has to defend itself. Does an internal function accept a broad dictionary even though every valid caller supplies the same fields? Is validation split between the route, a helper, and a component? Do two modules both believe they own the same decision?
Pick one representative flow and trace it from input to output. Record where the input is checked, where its type changes, and which modules make decisions about it. An agent can help assemble this map, but you still need to review whether its proposed grouping matches the behavior.
What will the abstraction let you delete?
Before building a shared abstraction, ask the deletion test: what existing code will this let us remove?
Suppose a proposed module centralizes validation. You should be able to point to checks and assertions that callers will no longer need. If every caller keeps its old defensive code and also has to understand a new wrapper, investigate why the new interface hasn’t simplified anything.
Deletion isn’t the only reason to introduce an abstraction. A new boundary might enforce access control or isolate a dependency. But for a cleanup justified by duplication, it’s reasonable to ask which duplication will actually disappear.
Our experiment found repeated view-screen implementations across apps and templates. We’d still need to compare their behavior before deciding what to share. Similar code may have different visibility rules, payloads, or state. Sharing the wrong part can leave everyone configuring exceptions.
Make the before and after concrete in your proposal: which responsibility moves, what callers stop doing, what must remain different, and how you’ll verify the behavior. For help structuring that investigation, see Matt Pocock’s architecture workflow.
Then try the change on one representative path. You’ll learn more from a working path with simpler callers than from a diagram of a repository-wide abstraction nobody has used yet.
Run the behavior you changed #
Before editing, reproduce the flow you expect to preserve. Knowing that it worked beforehand helps distinguish a cleanup regression from a problem that was already there.
Afterward, run the static checks and focused tests. Then inspect the requirement you wrote for the repair. The Analytics example passed its existing tests because those tests didn’t establish the missing validation guarantee.
Add a test where that guarantee can fail. If malformed input must be rejected before dispatch, exercise malformed input and verify that dispatch doesn’t happen. A test that supplies valid input and checks a successful response answers a different question.
When the change crosses modules, let those modules run together. When it affects something a user does, perform that flow in the actual interface. If deployment changes authentication, configuration, or persistence, verify it in the relevant environment too.
Check what your mocks are hiding
Mocks need a specific job in this process. A mocked payment provider can help you test what happens when a payment is rejected without relying on a live service. A test that mocks your own payment handler, persistence layer, and authorization code may skip the interactions most likely to break.
Which interactions does this test still exercise after we replace those modules? Keep focused tests where they help, and add integration coverage for the agreements between modules.
For a checkout change, actually place an order in the appropriate test environment. Check that the request reaches the payment integration, that failure produces the expected state, and that success is persisted correctly. A button showing “Success” is only one part of that behavior.
For an internal refactor, the equivalent might be exercising the public function with representative inputs and checking its outputs and side effects.
Add checks as you learn #
Once you’ve repaired a recurring problem, decide how the next person or agent will encounter the lesson.
A note in a guide can explain the reasoning. An executable check can point directly at the next occurrence during development. You don’t have to rely on the agent remembering which document discussed it.
This is, for example, how we’ve been building Agent Native. We add scripts when a problem becomes concrete, and modify them as the repository changes. At the revision used in our experiment, the repository already had formatting, type-aware Oxlint, TypeScript checks, and 66 repository-specific guard checks.
We didn’t need to invent that entire setup before building the product. Agents can make broad changes quickly, including changes to the tools they use. If a recurring mistake becomes easy to recognize, you can add the check then.
Make a failed check tell the agent what to do
Choose the kind of check that matches the failure. A lint rule can catch a forbidden import or an assertion pattern. A test can establish how invalid input behaves. An integration check can exercise whether modules still agree. Some architectural judgments will continue to need review.
Make failures explain the next step. “Forbidden pattern” sends the agent searching again. An error that identifies the existing helper or describes the required contract gives it a better chance of making the repair you intended.
Keep revisiting these checks. If the architecture changes, the rule may need to change with it. If agents keep satisfying the literal check while preserving the mistake, inspect both the repair instruction and what the check actually measures.
Start with one recurring problem #
Start with one behavior that keeps causing trouble. Inspect the relevant code, tell the agent what must become true, and run the behavior after the change. If the same mistake keeps returning, add a check that points to the repair. Keep the checks and abstractions only while they make the next change easier. You should be able to explain what got simpler and show that it still works.