Many Claude skill packs no longer help. Here's how to test yours with Anthropic's skill-eval plugin and know what to keep, trim, or delete.
Why Claude skills might be wasting your context right now #
A lot of the Claude skill packs people star on GitHub and install by default no longer do much. In one informal test, nine popular skill packs were run against a dummy file of the same length, and seven of the nine performed no better than the dummy. The reason is simple: Claude’s base models have absorbed most of the instructions those skills used to spell out. If your skill is still a long document of step-by-step rules for something like tone, formatting, or basic process, there’s a good chance the model already does that work without being told.
What changed since skills first became popular #
When Claude skills took off earlier in the year, the standard advice (including from a lot of the same creators now revising their guidance) was to write skills like a manual. You documented the entire process: every rule, every step, every edge case, often running to hundreds of lines. That approach made sense when the underlying model needed heavy guidance to produce decent output. A copywriting skill or a “humanize this text” skill would spell out exactly how to avoid robotic phrasing, because the model needed that steering.
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
That’s no longer true to the same degree. Newer Claude models apply a lot of that judgment on their own. They already avoid obvious AI-sounding phrasing in many cases and already apply sensible defaults for common writing and formatting tasks. A skill built as a full standard operating procedure is now mostly duplicating logic the model already has, while still taking up context space every time it loads.
How should you decide what belongs in a skill now? #
Anthropic’s own recommended approach is to flip the process. Instead of writing the skill first, try to complete the job with no skill at all. Then look at what Claude actually got wrong, missed, or handled in a way you didn’t want. Only those corrections go into the skill. A skill, under this model, is not a tutorial. It’s a short list of fixes for the specific things the base model doesn’t do correctly or doesn’t know about your situation (your house style, a client’s quirks, a non-obvious business rule). That list is almost always much shorter than the SOP-style skills people were building a year ago.
This matters because every line in a skill file is context the model has to read on every relevant run. A bloated skill isn’t just redundant, it’s actively competing for space with the parts of the conversation that actually need attention.
How does the skill-eval plugin work? #
Anthropic built a skill-eval plugin specifically to answer the “do I still need this” question without guesswork. The idea is straightforward:
- It runs the same job twice: once with your skill attached, once without it.
- You compare the two outputs side by side.
- If the no-skill version is just as good, the skill isn’t earning its keep anymore and can be cut or trimmed.
- If the skill version is better, you look at why , and keep only the parts of the skill responsible for that improvement.
Running this audit gives you one of two outcomes. Either you trim the skill down to just the corrections that still matter, cutting out instructions the model already follows by default, or you remove the skill entirely and free up that context for something else. Neither outcome means skills are dead. It means most existing skills are carrying dead weight that the current model generation made unnecessary.
Is it worth auditing skills you built months ago? #
Yes, especially anything built in the earlier wave of “skill engineering” advice that leaned on techniques like progressive disclosure and exhaustive rule-writing. Those techniques weren’t wrong for the models available at the time. But capability has moved fast enough that a skill tuned for an older model can now be mostly inert weight. The practical move is to treat skills as something you revisit periodically, not something you set once and forget. Given how quickly base capability is shifting, a skill that earns its place today might be redundant again within a few months.
Seven tools to build an app. Or just Remy. #
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
There’s also a lower-effort way to build corrected, trimmed-down skills from scratch: recording. Some Claude interfaces now support watching you actually do a task and generating a skill from what was observed, capturing only the corrections and niche knowledge that came up during the demo rather than a generic process doc. Skills built this way tend to come out noticeably smaller than hand-written ones, which tracks with the broader point: most of what used to go into a skill was already inside the model.
What else got swept up in the same “default is already good” shift? #
Skills aren’t the only area where the advice has reversed. A few related patterns worth knowing:
- Switching between models mid-session to save money (say, dropping from Opus to a cheaper model like Haiku for an easy question) can backfire, because each model keeps its own cache of the conversation. Switching models forces the full conversation history to be re-read at full price on the new model, and again when you switch back. Anthropic’s own guidance favors picking a model at the start of a job and adjusting effort level rather than hopping between models.
- A newer “advisor” feature lets a cheaper model do the main work while a stronger model gets consulted only at key moments (before committing to an approach, when the same error repeats, or before declaring a task done), without the cache penalty of a full model switch. It’s still marked experimental.
- Plan mode, once treated as essential, is viewed by some of Claude Code’s own creators as largely unnecessary now, with the reasoning that models need less hand-holding to form a workable plan on their own. The concept of planning (deciding what “done” looks like and what guardrails apply) still matters, especially for non-technical users, but the dedicated mode itself is less load-bearing than it used to be.
- Default settings in general (model choice, effort level) have gotten better than most manual overrides. Anthropic recommends starting most work on its higher-end model at medium effort, since medium effort on the newer model reportedly matches or beats high and max effort on the previous generation.
The common thread is that a lot of manual configuration people built up over the last year, whether skills, model-switching habits, or always-on planning steps, was compensating for model limitations that have since narrowed. The fix isn’t to abandon these tools, it’s to re-test assumptions against what the current model can already do unassisted.
Frequently Asked Questions #
What is a Claude skill, in current terms?
A skill is a set of instructions or files Claude loads for a specific recurring job. Earlier skills tended to be full written-out procedures. Current best practice treats a skill as a short list of corrections and niche context the base model doesn’t already know or doesn’t already do correctly.
How do I know if my skill is still useful?
Run the same task with and without the skill and compare the outputs. Anthropic’s skill-eval plugin automates this comparison. If the no-skill output matches the skill-assisted one, the skill isn’t adding value anymore.
Does trimming a skill actually matter for performance?
Yes. Oversized skills consume context space on every run, and unnecessary content in the context window has been shown to make it harder for models to track important details accurately. Trimming frees that space for the parts of a task that actually need it.
Should I delete all my old skill packs?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
No. The point isn’t to delete everything, it’s to test each one. Some skills still encode genuinely useful corrections or specialized knowledge the model doesn’t have. The goal is to stop assuming a skill is helping just because it used to.
Are these changes specific to Claude, or do they apply elsewhere?
The underlying principle, that base models have absorbed a lot of what used to require explicit instruction, applies broadly across tools like Cursor, Codex, and custom agent setups, not just Claude’s own interfaces.