{"slug": "will-claude-code-call-your-skill-on-its-own-38-runs-one-skill-seven-ways-to-it", "title": "Will Claude Code call your skill on its own? 38 runs, one skill, seven ways to describe it", "summary": "A developer ran 38 headless Claude Code 2.1.273 trials against a single commit-message skill, varying only how the skill was described, and found that auto-invocation hinged entirely on whether the one line shown in the skill listing contained the words of the user's request. A vague \"Helps with git stuff.\" description triggered the skill 0 times in 6 runs, while a specific description, the same vague line plus a when_to_use field, and a frontmatter-free file whose first line read \"# House commit format\" each triggered it 6 times in 6 runs. The author reports that when_to_use offered no advantage over a better description, and that a skill directory named commit-message partially compensated for a vague description on direct requests but not indirect ones.", "body_md": "We gave Claude Code one commit-message skill, changed only how it was described, and ran a small set of requests against it 38 times in headless mode. \"Helps with git stuff.\" was called 0 times in 6 runs; a specific description, the same vague line plus a `when_to_use` field, and a file with no frontmatter whose first line was \"# House commit format\" were each called 6 times in 6 runs. The deciding factor was whether the one line Claude sees in the skill listing contained the words of the request, not which field those words came from.\n\nThe skills docs say Claude \"uses this to decide when to apply the skill\" about the `description` field, and the troubleshooting section for a skill that does not trigger starts with \"Check the description includes keywords users would naturally say\". That is advice, not a number. We wanted to know how sharp the edge is: does a vague description make auto-invocation rarer, or does it switch it off? Does `when_to_use` do anything a better description would not? What happens when there is no description at all? We changed one thing at a time and counted.\n\nEverything below ran on Claude Code 2.1.273 on 2026-09-16, with the default model the CLI picked for the account, which the transcripts name as `claude-opus-5[1m]`. We ran 38 `claude -p` calls in total.\n\nEach run got a fresh directory from `mktemp -d` containing exactly one skill at `.claude/skills//SKILL.md` and a project settings file that turns off the skills that ship with Claude Code:\n\n```\n{\n \"disableBundledSkills\": true\n}\n```\n\nWe also started every run with `--setting-sources project`, so that nothing from our own `~/.claude` (user settings, user hooks, personal skills, plugins, our user-level `CLAUDE.md`) was loaded, and with `CLAUDE_CODE_DISABLE_AUTO_MEMORY=1`, so the runs did not write memory files. The point was a skill listing with one entry in it. The transcripts confirm that: every `skill_listing` attachment in them has `\"skillCount\": 1`.\n\nThe skill body was the same in every variant apart from one control. It describes a house format for commit messages, and it ends with a marker a model would not produce on its own:\n\n```\n# House commit format\n\nWrite the commit message in exactly this shape:\n\n- First line: `(): `, 60 characters or fewer, imperative mood.\n- Blank line, then one to three lines that say why the change was needed.\n- Last line: `Refs: none` unless the user gave a ticket id.\n```\n\nThe request always carried the same small diff: a `withRetry` helper that now rethrows an `AuthError` immediately and sleeps with exponential backoff between attempts. The wording in front of the diff came in three kinds:\n\nThe direct request uses the words \"commit message\". The indirect one asks for the same kind of output without those words. The negative one asks for something the skill is not for.\n\nThe skill's directory was named `kestrel` in every variant but one. A name like `commit-message` is itself a strong hint, and we wanted the description to be the only signal, so we used a word with nothing to do with git and tested the name separately.\n\nWe counted a run as \"invoked\" if the transcript under `~/.claude/projects/` contained an assistant `tool_use` block whose name is `Skill`. As a second check we looked for `Refs: none` in the final answer. The two signals agreed in all 38 runs: the marker appeared in exactly the 22 runs that called the skill and in none of the 16 that did not.\n\n| Variant (what the listing said) | direct | indirect | other | \n|---|---|---|---|\n| `description: Helps with git stuff.` | 0/3 | 0/3 | \"house format\" hint: 2/2 | \n| Specific description (below) | 3/3 | 3/3 | negative: 0/2 | \n| Vague description + `when_to_use` | 3/3 | 3/3 | negative: 0/2 | \n| No frontmatter, body starts `# House commit format` | 3/3 | 3/3 | | \n| No frontmatter, body starts `# Notes` | 0/2 | | | \n| Vague description, directory named `commit-message` | 2/2 | 0/2 | | \n| Specific description + `disable-model-invocation: true` | 0/2 | | | \n\nThe specific description was: \"Writes git commit messages in this repository's house format (type(scope): summary line, a short why, a Refs line). Use when the user asks for a commit message or wants a diff summarized for the git log.\"\n\nWe ran each main cell three times and the smaller controls twice. With counts this small we do not read the table as rates. What stands out is that no cell was mixed. Every variant either fired on every run of a given request or on none of them.\n\nWith the vague description, Claude answered all six direct and indirect requests itself, in one turn, without looking at the skill. The answers were reasonable commit messages. They were not in the house format: no `type(scope):` prefix, no `Refs:` line. A typical first line was \"Add exponential backoff to withRetry and fail fast on AuthError\".\n\nThis is the failure that matters in practice. Nothing errors, the answer looks fine, and the only sign the skill was skipped is that the output quietly ignores the convention the skill exists to enforce. If you only check \"did I get a commit message\", you will not notice.\n\nThe listing line Claude saw was `- kestrel: Helps with git stuff.` A request for a commit message is arguably \"git stuff\". Claude still did not treat that as a reason to open a skill it knew nothing else about.\n\n`when_to_use` rescued the same vague line\nThe third variant kept `description: Helps with git stuff.` and added one field:\n\n```\nwhen_to_use: When the user asks for a commit message, or asks to summarize a diff for the git log.\n```\n\nThat variant fired 6 times out of 6, the same as the specific description. The docs say `when_to_use` is \"Appended to `description` in the skill listing\", and the transcript shows exactly what that means:\n\nClaude Code joins the two fields with \" - \" into one line. From the model's side there is no separate field, just a longer description. So in our runs `when_to_use` did nothing that putting the same sentence in `description` would not have done. Where it may still help is in how you maintain the file: the \"what it does\" part and the \"when to use it\" part stay apart in the frontmatter. Both fields count toward the same 1,536-character limit on the listing entry, which the docs state in the frontmatter reference.\n\nThe docs say that if `description` is omitted, Claude Code \"uses the first non-empty line of the markdown content\". We tested that with a SKILL.md that had no frontmatter at all. Its first line was the heading `# House commit format`, and the listing read `- kestrel: House commit format`, with the `#` removed.\n\nThat skill fired 6 times out of 6, as reliably as the carefully written description. Three words were enough, and the control below points to \"commit\" as the word that did it.\n\nTo check that this was about the words and not about the missing frontmatter, we changed only the heading to `# Notes`. The listing became `- kestrel: Notes`, and the skill fired 0 times in 2 direct runs.\n\nThis is the result we find most useful. The rule is not \"write a long description\" or \"fill in `when_to_use`\". Claude matches the request against one short line, and that line has to contain words that point at the request. A heading can do that by accident, and a description can miss it just as easily.\n\nWith the directory renamed to `commit-message` and the vague description unchanged, the listing read `- commit-message: Helps with git stuff.` The direct request, \"Write a commit message for this diff.\", invoked it 2 times out of 2. The indirect request, \"Give me a one-line summary of this change for the git log.\", invoked it 0 times out of 2.\n\nThe name matched the literal words of one request and not the other. A descriptive name narrows the gap, but it only covers requests that use the name's own words. A description can list the other ways people ask for the same thing, and the specific description and the `when_to_use` line both did: they mention \"the git log\", and both fired on the indirect request every time.\n\nWe gave the vague `kestrel` skill one more request: \"Write a commit message for this diff in our house format.\" Nothing in the listing mentions a house format. The skill fired 2 times out of 2.\n\nThe second of those runs is worth describing. Claude first said it would \"check whether the house format is defined anywhere\", and tried a Bash command that listed the working directory and read `~/.claude/CLAUDE.md`. In `-p` mode with no allow rules, that command needed approval and was denied; the JSON output records it under `permission_denials`. Claude then called the only skill in the listing. So the vague skill was found, but only after Claude had tried to look for the format elsewhere. The only difference from the six vague runs that never fired was a phrase in the request that suggested a written convention existed somewhere.\n\nThe specific description and the `when_to_use` variant both got the negative request, which asks for an explanation of the diff for a teammate. Neither fired, 0 out of 4 runs in total, and each answer came back in one turn as a short explanation in prose. Making the listing line specific did not make the skill fire on nearby requests that were not about commits. Four runs is a small sample, and one negative wording is only one wording.\n\n`disable-model-invocation: true`: not even listed\nWith the specific description and `disable-model-invocation: true`, the skill fired 0 times in 2 runs. The transcripts for those two runs had no `skill_listing` attachment at all, while every other run had one. That matches the docs' table for this field: \"Description not in context\". A manual-only skill is not a skill Claude decides against using. As far as the model can tell, it does not exist until someone types its name.\n\nBoth answers were ordinary commit messages without the house format.\n\nEvery run that called the skill reported 3 turns in the JSON output (one reported 4: the run with the denied Bash call). Every run that did not call it reported 1. Duration went from 2.5–6.2 seconds without the skill to 6.9–13.6 seconds with it. The `total_cost_usd` field, which the output labels as list-price based, went from $0.030–$0.046 to $0.056–$0.077 per run. That range leaves out our very first run ($0.22), which wrote about 20,000 tokens to the prompt cache that later runs mostly read from. All 38 runs together came to $2.21 on that field.\n\nIn many runs Claude passed an `args` string when it called the skill, usually a restatement of the request. Our skill body has no `$ARGUMENTS` placeholder, and the transcripts show Claude Code appending an `ARGUMENTS:` line to the end of the loaded skill text instead, which is the behaviour the docs describe for that case.\n\nThe indirect request is where the skill changed the answer the most. The user asked for one line. All 9 indirect runs that called the skill still produced the full house-format message with the reason lines and `Refs: none`. Five of them gave the one-line summary first and then offered the full message. Four led with the full message and pointed out that its first line was the summary. None of them just returned one line. A skill that says \"write the commit message in exactly this shape\" does not bend to \"just one line\" by itself. If a skill should apply to part of a request, its body has to say so. That is a separate lesson from triggering, and a skill that fires more often makes it matter more.\n\nThe format also pushed something out. Four of the 16 runs that did not call the skill, including both manual-only runs, ended their commit message with a `Co-Authored-By` trailer. None of the 22 runs that called the skill did. The skill's shape ends with the `Refs` line, and Claude kept to it. We did not check what decided whether the trailer appeared.\n\nThe repository that runs our shop has ten project skills, and each description says when the skill should be used; one also lists trigger phrases. This experiment shows why that matters: the listing line is what Claude matches the request against, and in our runs a line either contained the request's words or the skill was ignored.\n\nThe check we now use on a new skill is the listing line itself, not the SKILL.md. Read the line as `- : - `, and ask whether the words people use for the task appear in it, including the indirect ones. If the skill has no frontmatter, the line is the first line of the body. A heading like \"Notes\" gives Claude nothing to match, and in our two runs with it, Claude never picked the skill on its own.\n\nThe runner below writes one variant into a fresh temp directory and runs one request. Swap the frontmatter to test other descriptions.\n\n``` bash\n#!/bin/bash\n# usage: ./run.sh