Our Developer Relations team at Postman runs on a Claude Code plugin. 19 skills handle the work we do every day: writing blog posts, staging content to WordPress, auditing competitor sentiment on Reddit, syncing meetup calendars, generating newsletters, tracking event registrations, hunting conference CFPs, and managing our entire editorial pipeline end to end.
The plugin had been running for months. Skills fired correctly. Output was good. Nobody was measuring the token cost.
When we finally ran a structured benchmark, we found 3 universal problems across all 19 skills, none of them had declared allowed-tools
, the worst single skill cost 11,400 tokens per invocation, and fixing everything saved 13,300 tokens per combined run — worth about 4 cents in API cost at today’s pricing. This post documents the methodology, the findings, every number from the before and after, and what the savings actually add up to in dollars.
How tokens actually work in a Claude Code plugin #
Before the benchmark results, it helps to understand where a plugin’s tokens go. There are 3 distinct cost categories, and they are not equally expensive.
Always-on cost is the most expensive token in the plugin. Every skill’s YAML description
field loads into the system prompt of every Claude Code session, regardless of whether the user invokes that skill. One verbose description is a tax every user pays, every session, whether they run that skill or not. This is the highest-leverage real estate in the entire plugin.
Per-trigger cost fires when Claude decides a skill is relevant. The entire SKILL.md
body loads into context. A skill with a 19 KB body costs roughly 4,760 tokens every time it runs, even if the user only needed 20% of its content.
Runtime cost accumulates during execution: tool output, async polling loops, and intermediate narration as the model works through a workflow.
There is also a 4th cost that sits outside the plugin itself: the MCP server’s tool schemas. The Postman MCP Server’s full mode exposes over 100 tools. On clients that eagerly load schemas, that’s 40,000 to 70,000 tokens before the user types anything. Recent Claude Code versions defer MCP tool , which largely neutralizes this cost, but it informed some decisions in our optimization pass.
The benchmark methodology #
We used the skillit plugin for Claude Code, which scores each SKILL.md
file across 6 dimensions on a 0–2 scale for a maximum of 12 points:
| Dimension | What it measures |
|---|---|
| Description quality | Is the always-on description tight and specific? |
| Body size and focus | Is the body lean, or padded with reference material? |
| Progressive disclosure | Does bulk content live in references/ files, loaded on demand? |
| Tool scoping | Is allowed-tools declared with an explicit list? |
| Frontmatter validity | Is the frontmatter complete and well-formed? |
| Prompting craft | Are instructions clear, imperative, and unambiguous? |
We ran the full audit in parallel, spinning up 19 scorer agents simultaneously with one command:
/skillit:skill-audit
Results came back in under 2 minutes. The scores exposed 3 systemic problems across the entire plugin.
Finding 1: none of the 19 skills had allowed-tools #
Every skill scored 0/2 on tool scoping. Not some. All of them.
The allowed-tools
field scopes which tools Claude can call during a skill invocation. Without it, the model has access to every tool in the session. That is a correctness problem as much as a security one. A skill that should only call WebSearch
and Write
can accidentally call Bash
, Edit
, or any MCP tool. In practice this means the model can wander into unrelated tools mid-workflow, and it means sub-agents and clients that resolve tool schemas from the allowlist have to load far more schemas than they need.
The audit also surfaced 3 latent permission bugs while fixing this. The blog-copyeditor
skill was instructed to write output files but had no Write
permission declared. The blog-wordpress-stage
skill ran Python scripts via Bash
with no Bash
in scope. Scoping allowed-tools
turned these silent mismatches into explicit declarations.
The fix is a 1-line change per skill, but identifying the right tool list for each one required reading every file. Some skills need very little:
allowed-tools: ["WebSearch", "Write"]
allowed-tools: ["Bash", "Read", "Write", "Edit"]
Finding 2: descriptions were bloated, and they load every session #
The combined always-on description footprint across 19 skills was 5,090 characters before the optimization pass. Several were badly oversized:
| Skill | Before (chars) | Sentences | Problem |
|---|---|---|---|
| sentiment-apitools | 637 | 7 | Full PLG-skew caveat, all tool names, usage recommendation — all body content |
| blog-wordpress-scheduler | 342 | 3 | Scheduling rules (Tue/Thu-first logic, 2-week windows, Mon/Wed fallback) in description |
| meetup-calendar | 314 | 4 | Enumerated 3 operating modes as separate sentences |
| blog-wordpress-stage | 245 | 4 | Step summary duplicating body content |
The sentiment-apitools
description alone was a 637-character, 7-sentence block that loaded into every session regardless of whether the user ever ran a sentiment analysis. After trimming all descriptions to 1–2 sentences covering capability and output artifact, the combined footprint dropped to 4,071 characters. That is a 20% reduction on costs paid by every user in every session.
Based on the 4 characters/token rule of thumb, the description trims alone saved approximately 255 tokens per session across the plugin.
Finding 3: inline Python scripts were the biggest token problem #
The per-trigger cost problem was worst in the skills that embedded complete Python scripts directly in their SKILL.md
bodies. Every invocation loaded every line of those scripts into context, whether or not that step of the workflow was reached.
** meetup-calendar** was the worst offender: 1,177 lines, 45.6 KB, with 5 full Python scripts embedded inline. The scripts covered Google Sheets JWT authentication, grid data parsing, Luma event fetching, fuzzy name matching, and stat syncing. Estimated per-invocation cost: 11,400 tokens.
** blog-wordpress-stage** had 9 Python code blocks across its 611-line body covering Google Doc conversion, markdown-to-HTML rendering, image up to the WordPress Media API, tag lookup and creation, and post staging.
** blog-wordpress-scheduler** embedded a shared authentication setup, a complete US public holiday calculation function, a 140-line Tue/Thu-first slot-finding algorithm, and a dashboard calendar writer across 608 lines.
** luma-stats** had a single 200-line Python script making up the majority of its 346-line body.
The pattern across all 4 is the same: implementation detail that the model only needs at the specific step where it runs was loaded upfront, for every invocation, whether that step was reached or not.
The fix: progressive disclosure with a references/ directory #
The repair pattern is straightforward. Implementation detail moves out of the SKILL.md
body into a references/
subdirectory. The body keeps only orchestration: clear numbered steps with a pointer to the reference file at the step that needs it.
This is the same principle documented in our internal token optimization findings for the Postman MCP plugin. Applied there to just 2 skills (postman-context
from 19 KB to 7.7 KB, saving ~2,800 tokens per trigger; generate-spec
from 10.6 KB to 7.2 KB, saving ~840 tokens per trigger). Applied here across 9 skills with 15 reference files extracted.
Before (from luma-stats
):
### Step 2: Write and run the data-fetch script
Write the following Python script to /tmp/luma-stats.py, then run it.
[200 lines of Python inline]
After:
python3 /tmp/luma-stats.py [filter-arg]
The script lives in references/luma-stats.py
. Claude reads it when it reaches that step, not before. The always-loaded body dropped from 346 to 131 lines, a 62% reduction.
Reference files extracted across the plugin:
| Skill | Files moved to references/ | Lines removed from body |
|---|---|---|
| blog-wordpress-stage | gdoc-to-markdown.py, wp-md-to-html.py, wp-upload-images.py, wp-check-post.py, wp-manage-tags.py, wp-stage-post.py | 358 |
| meetup-calendar | get-google-token.py, meetup-parse.py, luma-fetch.py, meetup-match.py | 351 |
| blog-wordpress-scheduler | wp-shared-setup.py, wp-write-calendar.py, wp-find-slot.py | 242 |
| luma-stats | luma-stats.py | 215 |
| blog-write | final-checklist.md, post-types.md | 109 |
| newsletter-agentsandapis | example-newsletter.md, quality-checklist.md | 87 |
| cfp-hunter | personas.md, cfp-sites.md | 69 |
| event-sponsorships | classification.md | 53 |
| blog-dashboard-cleanup | state-file-ops.md | 44 |
| blog-wordpress-stats | fetch-wp-posts.py | 38 |
Results: full benchmark data #
All 19 skills, before and after. Token estimates use the 4 characters/token rule of thumb applied to raw file size.
| Skill | Grade | Before (~tokens) | After (~tokens) | Saved | Reduction |
|---|---|---|---|---|---|
| blog-wordpress-stage | 6→11 | 6,050 | 3,400 | 2,650 | 44% |
| meetup-calendar | 3→7 | 11,400 | 8,775 | 2,625 | 23% |
| blog-wordpress-scheduler | 6→11 | 5,975 | 3,900 | 2,075 | 35% |
| luma-stats | 7→11 | 2,810 | 1,165 | 1,645 | 59% |
| sentiment-apitools | 5→9 | 6,200 | 4,930 | 1,270 | 20% |
| blog-write | 8→10 | 8,175 | 7,305 | 870 | 11% |
| event-sponsorships | 8→11 | 1,765 | 1,100 | 665 | 38% |
| cfp-hunter | 6→10 | 1,210 | 650 | 560 | 46% |
| newsletter-agentsandapis | 5→10 | 2,700 | 2,180 | 520 | 19% |
| blog-wordpress-stats | 8→11 | 1,125 | 905 | 220 | 20% |
| blog-dashboard-cleanup | 8→11 | 975 | 790 | 185 | 19% |
| blog-ideas | 7→9 | 1,570 | 1,620 | — | allowed-tools only |
| blog-copyeditor | 7→9 | 2,940 | 3,020 | — | allowed-tools only |
| blog-create-from-gdoc | 7→9 | 2,035 | 2,045 | — | allowed-tools only |
| blog-prod-updates | 7→9 | 2,625 | 2,710 | — | allowed-tools only |
| blog-header-image | 8→10 | 6,765 | 6,775 | — | allowed-tools + bug fix |
| influencer-autoagent | 6→8 | 3,625 | 3,720 | — | allowed-tools + rubric fix |
| social-media-manager | 8→10 | 2,325 | 2,400 | — | allowed-tools only |
| blog-pipeline | 10→12 | 880 | 890 | — | allowed-tools only |
Total body savings per combined invocation of all 19 skills: 13,300 tokens (23% overall)
Always-on savings: 255 tokens per session, paid back by every user on every session whether or not they run any of the optimized skills.
Average grade: 6.5/12 → 9.5/12
What this saves in dollars #
Token counts convert directly to API cost. Pricing per Claude Sonnet 5’s standard input rate — $3 per million tokens — is what these savings are worth per invocation:
| Skill | Tokens saved | $ saved per invocation |
|---|---|---|
| blog-wordpress-stage | 2,650 | 0.80¢ |
| meetup-calendar | 2,625 | 0.79¢ |
| blog-wordpress-scheduler | 2,075 | 0.62¢ |
| luma-stats | 1,645 | 0.49¢ |
| sentiment-apitools | 1,270 | 0.38¢ |
| blog-write | 870 | 0.26¢ |
| event-sponsorships | 665 | 0.20¢ |
| cfp-hunter | 560 | 0.17¢ |
| newsletter-agentsandapis | 520 | 0.16¢ |
| blog-wordpress-stats | 220 | 0.07¢ |
| blog-dashboard-cleanup | 185 | 0.06¢ |
That’s fractions of a cent per call — these are input tokens shaved off a SKILL.md
body, not a full agentic run. Run every skill once (the 13,300-token combined total) and the body savings alone come to about 4 cents. The always-on description trim (255 tokens) is worth roughly 0.08 cents, but it’s charged on every single session in the plugin, whether or not any skill runs at all.
At this team’s usage — a handful of people running Claude Code through the workday — that’s on the order of a dollar or two a month, not a number worth a press release. What scales is session count: run the full pipeline once a day for a month and body savings alone are worth about $1.20; add the always-on tax across dozens of sessions a day and it climbs from there, linearly with usage. The dollar figure is modest at this volume. The 23% smaller context footprint on every call is the part that matters regardless of scale — it’s headroom the model spends on the actual task instead of on skill overhead.
(Pricing as of this writing: Claude Sonnet 5 standard rate, $3/1M input tokens. An introductory rate of $2/1M applies through 2026-08-31, which would lower every figure above by a third.)
What this means for building your own skills #
5 rules I would apply from day one on any new skill, based on what this audit cost to retrofit:
Set allowed-tools when you write the skill, not later. Identifying the correct tool list for 19 existing skills required reading every file. Write it as you go and it’s a 30-second decision instead of an hour of archaeology.
Treat descriptions as the most expensive real estate in the plugin. They load every session, for every user. The ceiling is 2 tight sentences: what the skill does and what it produces.
Scripts longer than 20 lines belong in references/. The
SKILL.md
body should say “read references/myscript.py
and run it,” not contain the entire implementation. Output templates, scoring rubrics, persona taxonomies, and example documents all follow the same rule.Descriptions that score thin need expanding, not just trimming. The cfp-hunter
skill had a 75-character single-sentence description. The scorer flagged it because the output artifact, the search constraints, and the argument hint were all missing. We expanded it to 203 characters. The fix doesn’t always go the direction you expect.
Make async polling cheap. Any workflow that returns HTTP 202 and requires polling should specify backoff (2s, 4s, 8s) and instruct the model to report only the final outcome, not narrate every poll. Left unspecified, the model happily narrates every round-trip and fills context with intermediate status messages.
Running the audit on your own plugin #
/skillit:skill-audit
/skillit:skill-optimize skills/my-skill
/skillit:skill-optimize --all
/skillit:skill-validate skills/my-skill/SKILL.md
After optimizing, check /context
and /cost
in your Claude Code session to see the real footprint change. The token estimates here use the 4 characters/token rule of thumb. Actual savings depend on Claude’s tokenizer and content type — code tokenizes differently from prose.
The audit for 19 skills ran in under 2 minutes. The optimization pass took longer: reading files, extracting reference content, and writing new files. But the scoring gave a prioritized list sorted worst grade first, so time went to meetup-calendar
at 3/12 and 45.6 KB before it went anywhere near blog-pipeline
at 10/12 and 3.5 KB.
Resources #
Claude Code documentationClaude Code skills referencePostman DevRel Claude Code skills pluginAnthropic token counting documentationPostman MCP Server