{"slug": "the-model-didnt-get-dumber-my-agent-skills-got-stale", "title": "The Model Didn’t Get Dumber. My Agent Skills Got Stale.", "summary": "A developer found that their AI coding agents performed worse after upgrading to newer models like Claude Opus 5 and GPT-5.6, but an audit revealed the issue was stale custom prompts, not model regression. The developer cleaned up outdated instructions and saw improved results, highlighting the importance of recalibrating prompts during model migrations. Anthropic and OpenAI both recommend such recalibration, with OpenAI reporting 10-15% score improvements from leaner prompts in internal evaluations.", "body_md": "When Claude Opus 5 and GPT-5.6 arrived, I expected my coding agents to become noticeably better.\n\nInstead, some of my workflows felt worse.\n\nThe agents seemed more eager, less predictable, and occasionally “dumber” than before. Naturally, I blamed the new models.\n\nVery scientific of me.\n\nThen I watched Andrej Karpathy’s interview on the No Priors podcast. One idea stuck with me: when an agent fails, the capability may already exist. The problem could be how we instruct it, what memory we provide, or how we arrange the workflow.\n\nThat made me question something I had mostly ignored:\n\n**Were my custom skills still compatible with the newer models?**\n\nI gave my agent this prompt:\n\nCan you audit our custom skills against the current models? Flag stale prompts, conflicting instructions, outdated assumptions, and anything that should be simplified or removed. Then test each skill on a representative task and propose the smallest updates needed.\n\nThe audit found instructions written around the behavior of older models. Some were redundant. Some were no longer necessary. Others pushed the newer models too hard and caused them to overdo tasks.\n\nAfter cleaning those up and testing the skills again, the results felt noticeably better.\n\nThis was not proof that every disappointing result is caused by an outdated prompt. Models can still regress, behave inconsistently, or introduce genuine breaking changes.\n\nBut both Anthropic and OpenAI recommend recalibrating instructions during model migrations.\n\nAnthropic’s Claude Opus 5 documentation says the model now verifies its work without being told. It specifically recommends removing verification instructions carried over from earlier models because they can cause over-verification.\n\nOpenAI’s GPT-5.6 guidance recommends removing repeated instructions, simplifying tool descriptions, and running the same evaluations after each change. In OpenAI’s internal coding-agent evaluations, leaner system prompts improved scores by roughly 10–15% while reducing token usage. OpenAI describes those numbers as directional and recommends validating them against your own workload.\n\nResearch also shows that prompt performance does not transfer perfectly between models. An ICLR 2024 study found that performance across prompt formats correlated only weakly between the models it evaluated.\n\nIn other words, a prompt that helped yesterday’s model may confuse tomorrow’s.\n\nMy new model-update checklist is simple:\n\nThis is basically dependency maintenance, except the dependency is natural-language behavior.\n\nCustom skills are not permanent documentation.\n\nThey are part of the agent system, and that system changes whenever the underlying model changes.\n\nBefore concluding that a new model has become worse, audit the instructions surrounding it. You may have upgraded the engine while keeping the old owner’s manual.\n\nOr, in developer terms:\n\n**The model might be fine. Your prompts may just have technical debt.**\n\n*Disclosure: I used OpenAI Codex to help organize and edit this article. The experience, observations, conclusions, and final review are mine.*", "url": "https://wpnews.pro/news/the-model-didnt-get-dumber-my-agent-skills-got-stale", "canonical_source": "https://dev.to/evbermudez/the-model-didnt-get-dumber-my-agent-skills-got-stale-8cm", "published_at": "2026-08-16 15:16:30+00:00", "updated_at": "2026-08-16 15:42:22.110608+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "developer-tools"], "entities": ["Claude Opus 5", "GPT-5.6", "Anthropic", "OpenAI", "Andrej Karpathy", "No Priors", "OpenAI Codex", "ICLR 2024"], "alternates": {"html": "https://wpnews.pro/news/the-model-didnt-get-dumber-my-agent-skills-got-stale", "markdown": "https://wpnews.pro/news/the-model-didnt-get-dumber-my-agent-skills-got-stale.md", "text": "https://wpnews.pro/news/the-model-didnt-get-dumber-my-agent-skills-got-stale.txt", "jsonld": "https://wpnews.pro/news/the-model-didnt-get-dumber-my-agent-skills-got-stale.jsonld"}}