The Model Didn’t Get Dumber. My Agent Skills Got Stale.

A developer found that their AI coding agents performed worse after upgrading to newer models like Claude Opus 5 and GPT-5.6, but an audit revealed the issue was stale custom prompts, not model regression. The developer cleaned up outdated instructions and saw improved results, highlighting the importance of recalibrating prompts during model migrations. Anthropic and OpenAI both recommend such recalibration, with OpenAI reporting 10-15% score improvements from leaner prompts in internal evaluations.

When Claude Opus 5 and GPT-5.6 arrived, I expected my coding agents to become noticeably better. Instead, some of my workflows felt worse. The agents seemed more eager, less predictable, and occasionally “dumber” than before. Naturally, I blamed the new models. Very scientific of me. Then I watched Andrej Karpathy’s interview on the No Priors podcast. One idea stuck with me: when an agent fails, the capability may already exist. The problem could be how we instruct it, what memory we provide, or how we arrange the workflow. That made me question something I had mostly ignored: Were my custom skills still compatible with the newer models? I gave my agent this prompt: Can you audit our custom skills against the current models? Flag stale prompts, conflicting instructions, outdated assumptions, and anything that should be simplified or removed. Then test each skill on a representative task and propose the smallest updates needed. The audit found instructions written around the behavior of older models. Some were redundant. Some were no longer necessary. Others pushed the newer models too hard and caused them to overdo tasks. After cleaning those up and testing the skills again, the results felt noticeably better. This was not proof that every disappointing result is caused by an outdated prompt. Models can still regress, behave inconsistently, or introduce genuine breaking changes. But both Anthropic and OpenAI recommend recalibrating instructions during model migrations. Anthropic’s Claude Opus 5 documentation says the model now verifies its work without being told. It specifically recommends removing verification instructions carried over from earlier models because they can cause over-verification. OpenAI’s GPT-5.6 guidance recommends removing repeated instructions, simplifying tool descriptions, and running the same evaluations after each change. In OpenAI’s internal coding-agent evaluations, leaner system prompts improved scores by roughly 10–15% while reducing token usage. OpenAI describes those numbers as directional and recommends validating them against your own workload. Research also shows that prompt performance does not transfer perfectly between models. An ICLR 2024 study found that performance across prompt formats correlated only weakly between the models it evaluated. In other words, a prompt that helped yesterday’s model may confuse tomorrow’s. My new model-update checklist is simple: This is basically dependency maintenance, except the dependency is natural-language behavior. Custom skills are not permanent documentation. They are part of the agent system, and that system changes whenever the underlying model changes. Before concluding that a new model has become worse, audit the instructions surrounding it. You may have upgraded the engine while keeping the old owner’s manual. Or, in developer terms: The model might be fine. Your prompts may just have technical debt. Disclosure: I used OpenAI Codex to help organize and edit this article. The experience, observations, conclusions, and final review are mine.