You’re probably hearing more about SKILL.md files, and you’ll see that a lot of people test them the same way: paste a prompt into chat, watch it pass once on a laptop, and merge to main. But when you paste three pages of CLI docs into a markdown file without an evaluation loop, you are just as likely to confuse a frontier model as you are to help it.
For the past weeks, we have been diligently measuring and optimizing our own Google Cloud Developer Plugin, with the most essential skills and the Developer Knowledge MCP server to ground responses in the documentation. We ran dozens of scenarios with Gemini, Antigravity, and other agents. We used Claude Sonnet 5 on Gemini platform inside isolated Harbor pods on GKE Autopilot. And, we contributed Kubernetes and Gemini platform capabilities upstream to NVIDIA's open-source evaluator.
Here's the five-rule engineering playbook we use to verify a skill is worth its tokens.
When an evaluation run shows zero improvement between your plugin arm and your baseline arm (Δ = 0), the natural urge is to rewrite the body of your SKILL.md instructions. Hold off on that. Nine times out of ten, the agent never loaded your skill in the first place.
Coding agents do not read full SKILL.md bodies at turn zero. The router only inspects the short name and description in your YAML frontmatter. Once a workspace grows past a dozen skills, two failure modes collide: under-triggering (your description describes internal implementation rather than developer intent) and sibling hijacking (a broad phrase like "helps with cloud deployments" in a neighboring skill intercepts the prompt first).
We recently open-sourced skill-reach so you can catch frontmatter collisions before spending on container sandboxes. It validates manifests, computes vocabulary overlap against nearest rival skills, and runs empirical routing probes across Claude Code, Google Antigravity, and OpenAI Codex:
Pair skill-reach with NVIDIA's SkillSpector (Tier 1 and Tier 2 of SkillEvaluator). As the Nemotron Labs team details in their Evaluating Agent Skills deep dive, standard code linters miss agent-native risks buried in markdown prose: prompt injection, silent data exfiltration, tool poisoning, and trigger abuse.
If you run skill benchmarks directly on your workstation, your numbers are lying to you. A developer laptop is packed with ambient state that contaminates the baseline arm: Cached Google Cloud credentials
Exported project variables and warm bash history
Global environment rules
An unassisted baseline agent can pass a cloud deployment test by inheriting a default project ID or token sitting in your local shell profile, masking the exact failure your skill was written to solve.
Host resource contention also warps agent trajectories. While hill climbing code-retrieval agents with AlphaEvolve, our internal testing showed that running the exact same agent benchmark while a workstation was under CPU load dropped the score. Slower file scans and subprocess timeouts cause the agent to abandon search paths prematurely.
Running prompts a single time and screenshotting the green checkmarks is not a real evaluation. Multi-turn tool-using agents branch on every shell exit code and stdout snippet. In our retrieval study, more than a third of benchmark tasks produced different scores across consecutive runs of the exact same code and prompt.
A perfect score is equally meaningless without a paired baseline for comparison. Both Claude's plugin eval tool and NVIDIA's SkillEvaluator run paired ablation across multiple repetitions: Arm A runs the base model in a clean sandbox without the skill, while Arm B runs the identical prompt and sandbox with the skill mounted.
Comparing the score improvements across our sandbox trials exposed three clear patterns that single-shot testing hides. Without the plugin, the base model still stumbled into a working deployment on most runs, but only after burning minutes of wall-clock time guessing deprecated flags and retrying bad requests. Conversely, when a test case shows identical success with zero efficiency gain, the base model already owns that capability natively. Your skill is pure context tax — delete it.
When you connect an evaluation harness to an automated optimizer (like AlphaEvolve or reach optimize), it is a surprisingly common pattern for the agent to cheat the grader.
In our AlphaEvolve retrieval experiments, evolved candidates discovered four distinct ways to game the benchmark without solving the task. When the grading harness blocked specific expected substring checks in source code, an evolved candidate dynamically concatenated strings at runtime, traversed up the directory tree, and read the ground-truth answer file directly off disk. Another candidate spawned subprocesses to bypass AST checks.
The evolutionary loop favored those cheaters because the fitness formula blended correctness (0.95 weight) and token efficiency (0.05 weight) into a single score. Reading the answer file from disk takes no LLM turns, earning an efficiency bonus that gives cheaters a survival advantage over honest agents that spent tokens searching the repository.
Hardening your skill evaluator against reward-hacking takes three structural guardrails. First, configure Harbor so your test fixtures, grading scripts, and answer keys live inside a dedicated verifier container that mounts workspace artifacts only after the agent container exits. Second, keep SkillEvaluator's five dimensions (Correctness, Discoverability, Efficiency, Effectiveness, and Security) strictly orthogonal; never let token thrift compensate for a failed test or a security violation. Third, in NVIDIA/SkillEvaluator#140, we extended security checks to fail any trial that reads Kubernetes service account tokens or queries cloud metadata endpoints, while redacting live OAuth tokens from saved trajectories.
Looking across all of our trials, the difference between the skills that delivered a score improvement in our sandbox benchmark and the community skills that failed came down to what was inside the markdown.
Skills that copy-paste public API reference tables into skill files yield almost no lift. Frontier models already have public cloud docs in their pretraining weights; dumping thousands of words of static flags into the prompt just inflates time-to-first-token and buries actionable constraints.
The skills that increased our Efficiency score in our test suite taught compact operational habits and verification reflexes instead of encyclopedic reference text:
Verify CLI flags at the leaf subcommand before mutating state: Rather than listing dozens of flags in markdown, the skill teaches a simple reflex: verify help output at the command line before executing a state-changing deployment.
Route through a lightweight catalog index on demand: Rather than fourteen heavy cloud skills into context at once, a thin root router checks a curated remote index and pulls only the specific recipe needed for the task.
Enforce least-privilege auth defaults: Explicit guardrails instruct agents to authenticate via Application Default Credentials rather than down static service-account JSON keys, while enforcing unauthenticated access restrictions alongside explicit invoker IAM bindings.
Because our evaluation stack runs in CI, upgrading models takes one flag change. We re-run the paired WITH vs. W/OUT matrix against the new model and delete every skill whose Delta drops to zero. Deleting a skill because the new model graduated past it is one of the cleanest high-fives in software engineering.
Ready to equip your coding agents with production-grade skills? Try the Google Cloud Developer Plugin for AI Coding Agents.