Getting a Gemma 4 model serving on a Cloud TPU involves a surprising amount of ceremony: finding a zone that actually has v6e capacity, requesting flex-start VMs, sizing boot disks, wiring Hugging Face tokens through Secret Manager, picking the right vLLM flags for TPU, and remembering to tear everything down before the billing meter embarrasses you.
I packaged all of that into ** tpu-management** — a Claude Code skill plus an MCP server (
tpu-devops
) — so you can just ask Claude to do it.The skill teaches Claude the full TPU serving lifecycle, and the MCP server gives it ~40 tools to execute it:
find_tpu
), check dimensioned quotas (get_zones_with_available_quota
), estimate cost before creating anything.vllm/vllm-tpu:nightly
, fetches the HF token from Secret Manager, and serves the model automatically.analyze_cloud_logging
), where the self-hosted model analyzes its own infrastructure's logs.vllm bench serve
against the deployment and get TTFT/throughput/P95 numbers back.It also encodes the hard-won details you'd otherwise learn at 2 a.m.: the default 10 GB boot disk that the vLLM TPU image overflows, the TPUS_PER_TPU_FAMILY
quota that's invisible to gcloud regions describe
, why you watch the serial console instead of SSH, and the exact vLLM flags Gemma 4 needs on TPU (--tool-call-parser gemma4
, --disable_chunked_mm_input
, and friends).
The fastest path is the Claude Code plugin marketplace — one command gets you the skill and the MCP server:
/plugin marketplace add xbill9/tpu-skill-claude
/plugin install tpu-management@tpu-skill-claude
Prefer the classic routes? Clone and install:
git clone https://github.com/xbill9/tpu-skill-claude
cd tpu-skill-claude
make skill-install # skill only, all projects
./project-setup.sh --global # skill + MCP server registration
Or skip the clone entirely and unzip the packaged skill:
curl -L -o /tmp/tpu-management-skill.zip \
https://github.com/xbill9/tpu-skill-claude/raw/main/dist/tpu-management-skill.zip
mkdir -p ~/.claude/skills && unzip -o /tmp/tpu-management-skill.zip -d ~/.claude/skills/
You'll need an authenticated gcloud
CLI, the TPU API enabled, and a Hugging Face token (the agent's save_hf_token
tool stores it in Secret Manager for you). Configuration is all environment variables: GOOGLE_CLOUD_PROJECT
, MODEL_NAME
, ACCELERATOR_TYPE
, TENSOR_PARALLEL_SIZE
.
Once installed, you talk to Claude Code like this:
"Find me a v6e-1 with available quota and serve gemma-4-12B on it"
Claude checks existing resources first (the skill's rule: never create before checking what exists), sweeps zones for capacity, creates the VM with the proven flags, follows the serial console until vLLM logs "Application startup complete," and hands you a working OpenAI-compatible endpoint. Then:
"Benchmark it and tear it down when done"
The skill ships with a full reference guide (references/tpu-builders-guide.md
) covering the TPU Builders Program: flex-start zones per TPU family, quota metrics and how to request increases, and troubleshooting — so Claude can answer capacity and billing questions even without the MCP server connected.
The repo doubles as both a Claude Code plugin marketplace and a plain skill repo. A Makefile
keeps the three distribution formats (project skill, plugin layout, zip) in sync from one set of sources, and the MCP server is a single-file FastMCP app — easy to read, easy to fork for your own model/accelerator combos.
Issues and PRs welcome: https://github.com/xbill9/tpu-skill-claude