{"slug": "a-claude-code-skill-with-86k-stars-says-it-cuts-tokens-65-it-cut-my-bill-1", "title": "A Claude Code Skill with 86k Stars Says It Cuts Tokens 65%. It Cut My Bill 1%.", "summary": "A Claude Code plugin with 86,000 GitHub stars claims to cut token usage by 65% by making the AI speak tersely, but testing shows it only reduces output tokens by about 6% on average and cuts the total bill by just 1%. The discrepancy arises because output tokens, which the plugin targets, account for only about 1% of total tokens and 20% of costs, while the bulk of costs come from re-reading context.", "body_md": "There's a Claude Code plugin with 86,000 stars that saves you tokens by making the AI talk like a caveman. No filler, no pleasantries, just grunts. Fewer words out, smaller bill. Obvious, right?\n\nI tried it. The AI really did get terser. My bill didn't budge.\n\nHere's why — and it's the part actually worth knowing.\n\n## Not all tokens cost the same\n\nYou're billed for three kinds of token at wildly different prices. The words the model *writes back*\nare the dearest. The text you *send in* is cheaper. And *re-reading* context it has already seen is\nthe cheapest of all — but there's a mountain of it. On Sonnet that's **$15, $3, and $0.30 per million\ntokens** — output priced 5× the input and 50× the cache-read.\n\nHere's the mountain. Every turn, the model re-reads the whole conversation so far — your files, the\ntest output, the earlier back-and-forth — and then adds a few new words on the end. So a real session\nis almost entirely that re-reading. The words the model actually writes come to about **1% of the\ntokens**.\n\nToken economics\n\nOutput = 1% of the tokens\n\nPooled across my runs, two rulers\n\n1%\n\nOutput — what the skills cut\n\nA sliver of the volume. Now switch the ruler to dollars →\n\nOutput is ~1% of the tokens but ~20% of the bill — pooled across my runs; your cache-to-output mix moves it. Even a perfect 65% cut shaves ~13% off a fifth; the rest of the bill never moves.\n\nCaveman shrinks those words. But 1% of the volume, even priced ~50× more than the re-reading, works\nout to only about **a fifth of the cost** — and that fifth is the *only* slice the skill touches. Cut\nit to zero and four-fifths of your bill hasn't moved. That's the whole thing: caveman optimizes the\n1% you can see and leaves the 80% you can't.\n\nOn a subscription — Pro or Max, which is most of us — your \"bill\" is really a usage quota, not\ndollars. Same story, though: Anthropic says its\n[limits are cost-based](https://support.claude.com/en/articles/11647753-how-do-usage-and-length-limits-work),\nso the pricey output still weighs the most and there's still barely any of it. They don't publish the\nexact formula, so I measured dollars — the number I can pin down — but compressing a sliver of the\nvolume won't buy you much runway either.\n\n## So I measured the cost, not the token count\n\nI ran seven coding tasks — from a one-line helper to a multi-file refactor — with the skill and\nwithout, metering the real dollar cost of each run with ** vigiles**,\nan open benchmark harness I built to measure cost instead of token counts — then added up what each\nactually cost. It reruns on your own Claude subscription:\n\n[the same seven tasks, with the skill and without](https://github.com/zernie/vigiles/tree/main/bench/ecosystem).\n\nThe skill works: on most tasks the AI wrote less. But the output only shrank about **6% on average**,\nnot the 65% on the label. And the total cost landed **within 1%** of not using it at all. A real cut\nto the words, no change to what you pay. (I tried a second popular \"token-efficient\" skill too — its\noutput *grew*, and the cost went **up 10%**.)\n\n## ▶The full numbers (per-task, both skills, p-values)\n\nSeven tasks × five paired runs each = 140 runs on Sonnet (~$10 API-equivalent, $0 on my\nsubscription). Output change per task — negative means the skill cut output, positive means it grew —\nwith the Welch p-value; `*`\n\nis significant at p<.05, and the two Caveman cuts (.002, .006) also\nclear a Bonferroni correction for the seven tasks. \"The bill\" is pooled dollars across every run; the\noutput column is the mean of per-task ratios, so the two use different denominators and don't line\nup 1:1.\n\n| Task | Caveman outΔ | p | token-efficient outΔ | p |\n|---|---|---|---|---|\n| slugify | +54% | .27 | −2% | .78 |\n| debounce | −28% | .22 | −7% | .52 |\n| bugfix-offbyone | −31% | .002* | −6% | .87 |\n| bigO | −18% | .27 | +6% | .79 |\n| regex-email | −28% | .006* | +127% | .037* |\n| review-doc | −8% | .80 | +54% | .46 |\n| refactor-suite | +21% | .65 | +33% | .19 |\n\nPooled bill: **Caveman −1%** (flat), **token-efficient +10%**. Output is ~20% of the dollar cost\neither way, and every answer stayed correct. On the dollar side per task (the numbers carrying the\nthesis): Caveman's −1% is small moves both ways — a bugfix −16% (p=.03), the rest inside the noise.\ntoken-efficient's +10% is spread across tasks (regex-email +49%, review-doc +21%, refactor +9%,\nbigO +6%), not one outlier — debounce (−13%) is the only task that got cheaper. Full per-task dollar\ndeltas + p-values are in the JSON. Raw runs: [the 140-run JSON](/data/token-savings-runs.json),\nand the [ /caveman-compress test](/data/token-savings-compress-runs.json). Method + harness:\n\n[.](https://github.com/zernie/vigiles)\n\n`vigiles`\n\nOne exception, for completeness: the API's *per-minute* rate limits are a different axis from cost.\nFor most models cache-reads don't count toward the input-rate limit, and output has its own bucket —\nso if you're pinned against the output-per-minute ceiling specifically, trimming output does buy\nthroughput. That's a narrow case, not your monthly bill or quota.\n\n## Its other trick doesn't help either\n\nTo its credit, the skill knows the output is a small target, so it ships a second tool that compresses\nyour *instructions* file instead. I shrank one by 68% with it. The cost still didn't care — a\n500-token file is a rounding error next to a 100,000-token session.\n\nInput compression · /caveman-compress\n\nfile −68%, bill −14% (n.s.)\n\nIt compressed the conventions file 68%, beating its own 46% claim. The bill moved 14%, but that's inside the run-to-run noise: a 500-token file is a rounding error next to a 100,000-token session.\n\n## Where the savings actually are\n\nIf you want a smaller bill — or more runway before you hit your limit — don't squeeze the words. Look at what reloads every single turn:\n\n**Big files and command output you pulled in once.** A file you opened, a 10k-line test log, a sprawling`git diff`\n\n— read a single time, then silently re-read on every turn for the rest of the session. That reloaded pile, not the model's replies, is what quietly grows your bill.**A bloated conversation.** Hand work to subagents so the main thread stays small instead of dragging a growing context through every turn.**Never pruning.**`/compact`\n\nand trimming cut the big cheap pile directly — the pile that's almost all of your cost.\n\nNote the one that *used* to headline this list: unused MCP servers. Current Claude Code loads MCP\ntool definitions [on demand](https://code.claude.com/docs/en/mcp#scale-with-mcp-tool-search) instead\nof shipping every server's whole tool list each turn, so idle servers barely cost you now — the old\n\"prune your MCP list to save tokens\" advice is mostly obsolete unless you've pinned a server to always\nload. Which is the whole lesson again: the token math moved, so the folk wisdom is a year stale.\n\nI haven't benchmarked those the way I benchmarked the skill, so treat them as leads, not promises. But that's where the money is.\n\n## Before you install the next one\n\nAsk it one question: which tokens does it cut, and did anyone measure the actual *cost* — not the\ntoken count — on a real session? If the answer is \"it makes the model write less, measured on one\nprompt,\" you already know what it'll do to your bill. Maybe it works. Maybe the model really does\nsay less. But the words are a sliver of what you pay, and you'll barely feel it.", "url": "https://wpnews.pro/news/a-claude-code-skill-with-86k-stars-says-it-cuts-tokens-65-it-cut-my-bill-1", "canonical_source": "https://zernie.com/blog/token-savings-wrong-number/", "published_at": "2026-07-07 00:00:00+00:00", "updated_at": "2026-07-08 13:04:39.497010+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools"], "entities": ["Claude Code", "Anthropic", "Sonnet", "vigiles", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/a-claude-code-skill-with-86k-stars-says-it-cuts-tokens-65-it-cut-my-bill-1", "markdown": "https://wpnews.pro/news/a-claude-code-skill-with-86k-stars-says-it-cuts-tokens-65-it-cut-my-bill-1.md", "text": "https://wpnews.pro/news/a-claude-code-skill-with-86k-stars-says-it-cuts-tokens-65-it-cut-my-bill-1.txt", "jsonld": "https://wpnews.pro/news/a-claude-code-skill-with-86k-stars-says-it-cuts-tokens-65-it-cut-my-bill-1.jsonld"}}