Show a model your old code and it writes your old bugs: 32 runs, 0% reuse

A developer ran 32 controlled generations across four open-weight models (DeepSeek v4 Pro, Llama 4 Maverick, GLM 5.3, Mistral 3 14B) using the same prompt but swapping in two different commits of their own football quiz app repository. When shown the pre-migration commit, 100% of runs hand-rolled a new 190-line search component and reproduced the team's own historical defects; when shown the post-migration commit, 100% reused the shared component in a 41-line wrapper, with no run crossing over.

Last July I spent seven pull requests deleting the same component eleven times. Eleven games in my football quiz app had each grown their own search box, and they had drifted apart in the way duplicated code always does: arrow keys behaved four different ways, only four of the eleven kept the mobile keyboard down at the start of a round, several could scroll the page while you were arrowing through results, and one could submit a stale result. The migration replaced all of them with a single shared component, and the final pull request added a test that fails if a twelfth one ever appears. That gives me something most people writing about AI and code do not have: a repository where I know exactly what the right answer is, and where the wrong answer used to live at a known commit. So I asked a question I had been wondering about since I started letting models write parts of this app. Does the code that already exists decide what the model writes next? It does, completely. The setup is deliberately boring, because the whole point is that only one thing moves. I wrote out a task asking for a footballer search box on a new game screen, told the model to match the conventions of the code it was being shown, and then sent that same text over and over, eight times per model per condition, at a temperature of 0.7 so the runs would not all be carbon copies of each other. What changes between conditions is which commit of my own repository gets pasted in above the task. For the first I went back to 82385a5 , where nine per-game search components are sitting there and nothing shared exists yet, and for the second I used cc78828 , the commit that closed the migration, where the shared component is present and TeamTies is shown calling it. Both conditions get roughly four hundred lines of genuine source from the repository as it stood, so neither is winning on prompt size, which was the first objection I expected and wanted to rule out before I looked at anything else. The models were DeepSeek v4 Pro, Llama 4 Maverick, GLM 5.3 and Mistral 3 14B, an odd-looking lineup that has a dull explanation: Claude and GPT both come back 403 on my account's tier, so open weights are what I had to work with. | context shown | reused the shared search | hand-rolled a new one | median lines | |---|---|---|---| | the forked era | 0% | 100% | 190 | | after the migration | 100% | 0% | 41 | Thirty-two runs each, and not one crossed over. Same model, same instructions, 190 lines of hand-written state machine in one case and a 41-line wrapper in the other, decided entirely by what was lying around in the repository. I expected a tendency. I did not expect it to be unanimous across four models from four different labs. The interesting part is not the line count, it is what is in those 190 lines. The migration existed because eleven copies had drifted on specific behaviours, and the test I added lists them. So I checked the generated components against that same list. Of the 33 hand-rolled components the old context produced: SEARCH MIN QUERY LENGTH , and = 2 instead — the exact thing the repository's own documentation tells you never to do. These are not generic AI mistakes. They are my team's mistakes, from 2025, faithfully reproduced in 2026 by models that were shown the code containing them. The forked components taught the models to fork, and taught them the defects too. There is a cleaner way to put the damage. Every one of the 32 components generated from the old context calls useFootballerSearch . That hook was deleted in the final pull request of the migration. All 32 would fail to compile against the repository as it stands today, which is a strange thing to say about code that perfectly matches the codebase it was shown. If old code drags the model backwards, the practical question is what you can do about it when your repository is full of old code. I ran two more conditions to find out. In the first, the model sees only the forked components, exactly as before, plus one paragraph from the repository's CLAUDE.md : the rule saying every search goes through the shared utility and never hand-roll another typeahead. In the second, the model sees no source for the shared component at all, only its filename in a directory listing. | context shown | reached for the shared search | hand-rolled | |---|---|---| | forked code only | 0% | 100% | | forked code + the written rule | 100% | 0% | | just the filename in a listing | 97% | 3% | | the shared component's source | 100% | 0% | One paragraph of documentation, sitting next to nine hundred lines of exactly the code it forbids, flipped every single run. That is a better return than I expected from a file most people assume gets skimmed. Here is where the neat story breaks, and it took a second measurement to see it. Reaching for the shared search only means the model wrote the component's name into its JSX. It does not mean it called it correctly. The component has 23 real props, so I compared the props in each generated call against that list. When the model could see the component's source, it invented nothing: none of those 32 calls used a prop that does not exist. When it had only the written rule, 28.6% of calls invented at least one. When it had only the filename, 82.1% did, averaging 4.64 imaginary props per call. So the file listing result is largely a mirage. Telling a model that the component exists reliably stops it hand-rolling a typeahead, and then it writes a confident call to an API it has never seen, with onFootballerSelect and debounceMs and other things I never wrote. The 97% reuse figure is nominal; most of that code does not compile. The rule and the source do different jobs. The rule decides which road it takes. The source decides whether the code at the end of the road is real. I expected a single leftover fork to poison the well, so I ran a condition with the shared component present and one of the old forked files still sitting in view. It made no difference: every run still reused the shared component, at essentially the same length. That is worth saying plainly because it contradicts the tidy version of this post. Models are not dragged down by the worst code in the repository, they follow the dominant pattern. You do not have to sweep up every last bad example before your AI-assisted work improves. You have to make the good abstraction exist, and make it visible. What I got wrong was the prop-counting instrument, and I nearly published its output. My first pass reported that every call in every condition invented props that do not exist, including the condition where the model could read the component's source. I believed it for about a minute, because it flattered a story I was already enjoying. Then it occurred to me that a model with the type definition in front of it inventing nine imaginary props on every single run is not a plausible thing to happen. The regex that was meant to extract the real prop names had matched nothing, so the set of real props was empty and every prop the models used counted as invented. That result was measuring my own broken parser. With the list actually populated, the true figure for that condition is zero, and the real finding — that invention rises sharply as context thins — only appears once the instrument works. It is the same lesson as the last few of these: when a number arrives that is both extreme and convenient, the first suspect is the thing doing the measuring. If you are letting a model write code in an existing repository, the repository is your prompt, whether you intended it or not. Everything in these results points the same way. The model is not consulting best practice, it is pattern-matching on what is in front of it, and it will reproduce a pattern you abandoned a year ago with complete confidence and a hook that no longer exists. Three things follow, in the order they pay off. Delete the old pattern, because as long as it is the only thing in the repo it is the specification. Write the rule down where the tools will read it, since that one paragraph was worth more than every line of example code around it. And make sure the real implementation, not just its name, is reachable, because a model that knows your component exists but cannot see it will invent its interface without hesitating. The migration was worth doing for the humans. The measurements say it changed what the machines write too, by a wider margin than it changed what we write. Everything above ran on DigitalOcean's inference API against deepseek-v4-pro , llama-4-maverick , glm-5.3 and mistral-3-14B , 160 completions in total, eight per model per condition.